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ABSTRACT 


Plant  information  systems  provide  operators  with  information  supporting  situation  assessment,  monitoring  and 
detection  of  disturbances,  response  planning,  and  response  implementation.  The  importance  of  the  design  of 
information  systems  for  human  performance  and  reliability  has  long  been  recognized.  Recent  advances  in  design 
go  beyond  the  ‘one  sensor  -  one  indicator'  display  systems  of  conventional  plants.  Computer-based  systems 
provide  a  variety  of  ways  to  process  and  present  data.  These  characteristics  of  design  represent  a  trend  toward 
making  displays  more  immediately  meaningful  to  personnel  by  mapping  display  representations  to  important 
aspects  of  plant  processes  and  to  underlying  cognitive  mechanisms,  such  as  perceptual  processes  and  mental 
models.  The  objective  of  this  study  was  to  develop  guidance  for  human  factors  review  of  advanced  information 
systems  based  on  a  technically  valid  methodology  for  developing  guidance.  To  support  this  objective,  we 
developed  a  characterization  framework  for  describing  key  design  characteristics  of  information  systems.  The 
characterization  includes  the  following  major  components:  information  requirements,  representation  systems, 
interface  management  functions,  and  display  devices.  Representation  systems  were  further  hierarchically  divided 
into  display  elements  (basic  building  blocks  of  displays,  such  as  axes,  alphanumeric  elements,  and  icons),  display 
formats  (such  as  mimic  displays,  configural  displays,  and  novel  graphics),  display  pages  (a  defined  set  of 
information  that  is  intended  to  be  displayed  as  a  single  unit),  and  display  networks  (the  organization  of  display 
pages).  Then,  we  examined  research  in  the  following  areas:  (1)  generic  cognitive  tasks  that  an  information  system 
must  support,  (2)  information  requirements  analysis,  (3)  information  representation,  and  (4)  information 
organization.  This  research  was  used  to  provide  the  technical  basis  on  which  guidelines  for  review  of  design  were 
developed.  These  guidelines  address  both  the  design  process  and  the  implementation  of  advanced  information 
systems.  However,  there  were  aspects  of  information  systems  for  which  the  technical  basis  was  insufficient  to 
support  the  development  of  guidance.  These  were  identified  as  issues  to  be  addressed  in  future  research. 
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EXECUTIVE  SUMMARY 


The  Human-System  Interface  Design  Review  Guideline,  NUREG-0700,  Rev.  1  (O’Hara,  et  al.,  1996),  was 
developed  to  provide  guidance  on  human  factors  engineering  (HFE)  for  the  U.S.  Nuclear  Regulatory  Commission 
(NRC).  The  NRC  staff  uses  NUREG-0700  for  (I)  reviewing  submittals  of  human-system  interface  (HSI)  designs 
prepared  by  licensees  or  applicants  for  a  license  or  design  certification  of  a  commercial  nuclear  power  plant  (NPP), 
and  (2)  undertaking  HSI  reviews  that  could  be  included  in  an  inspection  or  other  types  of  regulatory  review  of  HSI 
designs,  or  incidents  involving  human  performance.  It  describes  those  aspects  of  the  HSI  design  review  process 
that  are  important  to  identifying  and  resolving  human  engineering  discrepancies  that  could  adversely  affect  plant 
safety.  NUREG-0700  also  has  detailed  HFE  guidelines  for  assessing  the  implementation  of  HSI  designs. 

In  generating  NUREG-0700,  Rev  I,  several  topics  were  identified  as  “gaps”  because  there  was  an  insufficient 
technical  basis  upon  which  to  formulate  guidance.  One  such  topic  is  the  integration  of  advanced  HSI  technology 
into  conventional  NPPs.  The  NRC  is  currently  sponsoring  research  at  Brookhaven  National  Laboratory  (BNL)  to 
(1)  better  define  the  effects  of  changes  in  HSIs  brought  about  by  incorporating  digital  technology  on  personnel 
performance  and  plant  safety,  and  (2)  develop  HFE  guidance  to  support  safety  reviews,  should  a  review  of  plant 
modifications  or  HSIs  be  necessary. 

Based  upon  the  literature,  interviews,  and  site  visits,  O’Hara  et  al.,  (1996)  identified  changes  in  HSI  technology 
and  their  potential  effects  on  personnel  performance.  The  topics  were  then  evaluated  for  their  potential  safety 
significance  (Stubler  et  al.,  1996);  information  systems  was  one  HSI  technology  that  was  found  to  be  potentially 
safety  significant. 

The  importance  of  information  system  design  for  the  performance  and  reliability  of  operators  has  long  been 
recognized  (Patrick,  1987).  In  conventional  control  rooms  (CRs),  the  large  spatially  fixed  arrangement  of  controls, 
displays,  and  alarms  supports  rapid  scanning  and  pattern-recognition  approaches  to  plant  diagnosis.  Information 
is  primarily  presented  by  detailed  parameter  indicators  on  a  variety  of  analog  devices,  such  as  gauges  and  linear 
scales. 

Computer-based  systems  provide  a  variety  of  ways  to  process  and  present  data.  Information  may  be  presented  as 
individual  parameter  displays  (as  in  conventional  systems)  or  in  processed  form;  e.g.,  raw  data  parameters  are 
integrated  to  provide  higher  levels  of  information.  These  design  trends  are  toward  making  displays  more 
immediately  meaningful  to  personnel  by  mapping  the  presentation  of  information  to  underlying  cognitive 
mechanisms,  such  as  perceptual  processes  and  mental  models.  However,  this  places  a  significant  burden  on  the 
designer  both  to  anticipate  the  information  needs  and  map  them  onto  appropriate  display  formats  that  correspond 
to  the  operator’s  mental  models.  This  is  because  designs  that  appeal  to  engineers  are  not  necessarily  well  suited  to 
plant  operations  and  maintenance  personnel.  If  mistakes  are  made  in  the  design,  then  their  use  may  impair  rather 
than  support  performance.  This  may  lead  to  the  initiation  of  an  event  or  complicate  operators’  responses  to  an 
event  by  causing  them  to  take  improper  actions  or  omit  necessary  actions. 

In  general,  poor  implementation  of  information  systems  can  lead  to  human  performance  problems.  Some  human 
performance  issues  associated  with  display  design  include  the  following:  Information  overload;  limited  display 
area;  lack  of  overview  of  plant  status,  appropriateness  of  display  formats  and  page  arrangements  to  operator  tasks; 
and  improper  integration  and  presentation  of  numerical  data. 

We  focused  our  efforts  on  those  aspects  of  information  system  design  that  are  most  relevant  to  the  nuclear  industry, 
where  the  research  is  most  active,  and  where  the  results  are  reflected  in  actual  NPP  design  efforts:  information 
requirements,  representation,  and  organization.  Information  requirements  refer  to  what  information  operators 
need  to  safely  and  efficiently  operate  the  plant.  This  is  a  vital  step  in  design  of  information  systems  and  recent 
approaches  to  its  analysis,  such  as  ecological  interface  design  approaches  that  largely  stem  from  the  nuclear  and 
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process  control  industries,  promise  to  enhance  the  analysis  of  information  needs  (e.g.,  Vicente  and  Rasmussen, 
1992). 

Information  representation  refers  to  how  the  needed  information  is  displayed  in  the  control  room.  The 
development  of  novel  graphical  means  to  represent  information  in  a  format  that  supports  immediate  and  accurate 
understanding  of  plant  conditions  has  a  long  history  in  the  nuclear  industry  stemming  from  the  use  of  simple 
polargraphic  displays  for  safety  information  (e.g.,  Woods,  Wise,  and  Hanes,  1981)  to  more  recent  efforts  to  . 
represent  the  thermodynamic  state  of  the  plant  using  graphic  displays  based  on  the  Rankine  cycle  (Beltracchi, 
1995). 

Finally,  structure  is  imposed  on  the  information  representations  by  organizing  them  into  display  pages  and  the 
pages  into  an  entire  display  network.  This  is  a  significant  issue  in  process  control  facilities,  because  the  networks 
may  contain  hundreds  or  even  thousands  of  display  pages  (O’Hara,  Stubler,  and  Higgins,  1996).  While  this  aspect 
of  information  system  design  has  been  generally  neglected,  recent  research  suggests  that  organization  is  significant 
to  operator  performance,  and  some  common  organizational  schemes  can  make  it  difficult  for  operators  to  perform 
their  tasks  (Heslinga  and  Herbert,  1995). 

The  objective  of  this  study  was  to  develop  HFE  review  guidance  for  information  systems  based  on  a  technically 
valid  methodology.  To  support  this  objective,  the  following  tasks  were  undertaken: 

•  Development  of  a  framework  for  characterizing  key  design  features  of  information  systems 

•  Development  of  a  technical  basis  using  research  and  analyses  on  human  performance  relevant  to  information 
systems 

•  Development  of  HFE  review  guidelines  for  information  systems  in  a  format  that  is  consistent  with  NUREG- 
0700,  Rev.  1,  and  NUREG-071 1 

•  Identification  of  remaining  information  systems  issues  for  which  research  was  insufficient  to  support  our 
development  of  NRC  review  guidance 

The  status  of  each  will  be  briefly  addressed  below. 

Characterization  Framework  for  Information  Systems 

Existing  information  systems  were  reviewed  to  identify  the  dimensions  and  characteristics  along  which 
information  systems  can  be  defined.  Characterization  was  important  because  it  provided  a  structure  within  which 
the  reviewer  could  request  information  about  a  system,  and  with  which  to  structure  the  guidance. 

The  characterization  of  information  systems  included  the  following  major  components:  information  requirements, 
representation  systems,  management  functions,  and  display  devices.  Representation  systems  were  further 
hierarchically  broken  down  into  display  elements  (basic  building  blocks  of  displays,  such  as  axes,  alphanumeric 
elements,  and  icons),  display  format  (such  as  piping  and  instrumentation  diagram  (P&ID)  displays  and  trend 
graphs),  display  pages  (a  defined  set  of  information  that  is  intended  to  be  displayed  as  a  single  unit),  and  display 
networks  (the  organization  of  display  pages). 

Technical  Basis  Development 

The  development  of  detailed  review  guidelines  began  by  collecting  technical  information  on  which  to  base  the 
guidance.  Research  in  the  following  areas  was  examined:  (1)  operators’  generic  cognitive  tasks  that  an 
information  system  must  support,  (2)  analyses  of  information  requirements,  (3)  representation  of  information,  and 
(4)  organization  of  information. 
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The  plant’s  information  systems  are  the  basis  for  operators  to  perform  their  role.  The  general  tasks  that  the 
information  system  must  support  include  situation  assessment,  monitoring  and  detection,  response  planning,  and 
response  implementation.  Considerations  of  the  information  processing  basis  needed  for  the  operators’  cognitive 
tasks  have  to  be  reflected  in  design  review  guidance  for  the  information  system.  The  way  in  which  these  needs  are 
reflected  in  designing  information  systems  is  in  analyses  of  information  requirements,  the  representation  of 
information,  and  the  organization  of  information. 

In  NPP  design,  a  systems-engineering  approach  was  recommended  for  identifying  information  requirements.  Plant 
functions  are  broken  down  based  on  performance  requirements  and  allocated  to  humans  and  machines.  Human 
functional  requirements  are  further  analyzed  to  better  define  the  information  requirements  of  task  performance. 

EID  was  identified  as  a  potential  additional  approach  that  addressed  following: 

•  Specification  of  information  requirements 

•  Organization  of  information  among  the  levels  of  an  abstraction-hierarchy 

•  Development  of  innovative  displays  to  show  functionality 

•  Addition  of  analytical  redundancy 

Each  of  these  aspects  of  EID  may  have  some  performance-enhancing  features.  However,  the  associated  research 
had  methodological  weaknesses  that  limited  the  conclusions  that  could  be  drawn. 

Once  information  requirements  are  identified,  they  must  be  represented  in  the  information  system;  that  is,  display 
formats  are  developed  to  communicate  the  information  to  the  operators.  The  success  of  representational  aids  was 
discussed  as  being  a  function  of  the  characteristics  of  the  plant,  the  operators,  and  the  representation  through 
which  the  operators  view  and  act  on  the  plant.  All  three  are  essential  parts  of  the  overall  system.  Approaches  to 
representation  include  separable  displays  (individual  parameters  displayed  in  unrelated  form),  integral  displays 
(individual  parameters  integrated  into  object  display  where  the  individual  ones  are  not  visible),  and  configural 
displays  (individual  parameters  are  displayed,  but  emergent  perceptual  features  are  created  by  the  arrangement  and 
organization  of  parameters). 

In  general,  some  research  support  was  found  for  the  representation  principles;  however,  most  of  the  studies  could 
not  be  generalized  due  to  the  use  of  simple  systems,  tasks,  and  inexperienced  participants.  Thus,  extending  the 
research  findings  to  complex  systems,  such  as  NPPs  must  be  done  with  caution. 

While  the  organization  of  the  display  page  and  network  are  important,  there  were  few  data  on  organizing  large, 
complex  information  systems.  Such  studies  are  complicated  due  to  the  interactions  between  informational 
organization  and  other  design  considerations,  such  as  the  display  area  available  and  interface  management 
functions.  Further,  some  research  seemed  to  contradict  current  HFE  guidance  and  current  nuclear  practice;  that  is, 
very  dense  displays  were  found  to  lead  to  better  performance,  were  preferred  by  operators,  and  the  industry  ’s 
common  practice  of  function-system  network  organization  was  found  to  be  difficult  for  operators  to  use  when 
performing  certain  tasks. 

HFE  Review  Guidelines 


Once  the  technical  information  was  assembled,  a  draft  set  of  guidelines  was  developed.  The  guidelines  were 
organized  and  specified  in  a  standard  format.  In  general,  guidelines  were  only  developed  for  those  aspects  of 
display  design  that,  in  our  interpretation,  are  supported  by  the  literature.  Many  research  studies  were  either  weak 
in  experimental  methodology  or  limited  in  generalizability  to  complex  systems.  This  situation  constrains  the 
development  of  new  HFE  guidelines. 
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While  there  was  a  sufficient  technical  basis  to  develop  a  detailed  design-implementation  guidance,  as  is  typical  in 
NUREG-0700,  several  limitations  in  the  technical  basis  were  identified,  and  so,  many  issues  remain  for  which 
typical  NUREG-0700  guidance  could  not  be  developed  (the  specific  issues  are  listed  in  the  next  section).  Until 
sufficient  information  is  available  to  support  guidance  development  in  these  areas,  they  can  be  addressed  for 
specific  information  systems  case-by-case  using  design  process  review'  guidance.  Thus  two  types  of  guidance  was 
developed:  design  process  review  guidance  and  HFE  design  review  guidance. 

The  HFE  design  process  guidelines  were  designed  in  the  standard  format  adopted  in  NUREG-071 1  (O’Hara,  et  al., 
1994).  The  guidelines  were  organized  into  the  following  sections: 

•  Operating  Experience  Review 

•  Function  and  Task  Analysis 

•  Human- System  Interface  Design 

•  Training  Program  Development 

The  HFE  design  guidelines  were  developed  in  the  standard  format  adopted  in  NUREG-0700,  Rev.  1 .  The 
guidelines  were  organized  into  the  following  sections: 

•  Guidelines  for  General  Display 

•  Display  Format 

-  Integral  Formats 

-  Configural  Display  Formats 

The  guidance  was  peer  reviewed  and  revised.  The  new  guidance  will  be  integrated  into  the  existing  guidance  on 
information  systems  in  NUREG-0700,  Rev.  1. 

Information  Systems  Issues 

Where  there  was  insufficient  information  for  the  technical  basis  upon  which  to  develop  valid  design  review 
guidance,  an  issue  was  defined.  There  were  several  human  performance  issues  associated  with  information 
systems.  The  issues  were  organized  into  three  topic  areas: 

•  Technical  Basis  Issues 

-  Lack  of  a  Well-Defined  EID  Process 

-  Lack  of  Specific  Representation  Guidance 

-  Evaluation  of  Operating  Experience 

-  Critical  Testing  and  Evaluation  of  EID  Concepts 

-  Evaluation  of  Displays 

•  Design  Review  Issues 

-  Task  and  Temporal  Considerations 

-  Volume  of  Information 

-  Density  of  Display  Information 

-  Operator  Use  of  a  Large  Span  and  Variety  of  Displays 

-  System  Complexity  and  Emergent  Features 

-  Perceptual  Resolution 

-  Elements  of  Configural  Display 

-  Effect  of  Instrumentation  Failures 

-  Organization  of  Information 

-  Integration  of  EID  Displays  into  Remainder  of  HSI 
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Operator-Related  Issues 

-  Training  and  Qualification  Implications 

-  Operators’  Acceptance 

-  Internal  vs.  External  Mental  Models 
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These  design  characteristics  represent  a  trend  toward  making  displays  more  immediately  meaningful  to  personnel 
by  mapping  the  presentation  of  information  to  underlying  cognitive  mechanisms,  such  as  perceptual  processes  and 
mental  models.  However,  this  places  a  significant  burden  on  the  designer  both  to  anticipate  the  information  needs 
and  map  them  onto  appropriate  display  formats  that  correspond  to  the  operator’s  mental  models.  This  is  because 
designs  that  appeal  to  engineers  are  not  necessarily  well  suited  to  plant  operations  and  maintenance  personnel.  If 
mistakes  are  made  in  the  design,  then  their  use  may  impair  rather  than  support  performance.  This  may  lead  to  the 
initiation  of  an  event  or  complicate  operators’  responses  to  an  event  by  causing  them  to  take  improper  actions  or 
omit  necessary  actions. 

In  general,  poor  implementation  of  information  systems  can  create  human  performance  problems,  as  shown  by  the 
NRC’s  experience  with  safety  parameter  display  systems  (SPDSs)  (Eckenrode,  1991;  Liner  and  DeBor,  1987). 
Modem  computer-based  information  systems  are  considerably  more  complex  than  the  SPDSs  of  the  1980s.  Some 
human  performance  issues  associated  with  display  design  include  the  following: 

Information  Overload- The  vast  quantity  of  information  that  is  available  through  the  display  system  may  impose 
high  levels  of  mental  workload.  For  example,  demands  on  the  operator’s  attention  may  be  high  (e.g.,  the  operator 
may  need  to  view  multiple  sets  of  information  at  the  same  time  or  in  rapid  succession).  Also,  the  need  to  consider 
the  relevance  of  a  vast  set  of  information  to  current  task  requirements  may  impose  high  demands  on  the  operator’s 
ability  to  analyze  information. 

Appropriateness  of  Display >  Formats  and  Page  Arrangements  to  Operator  Tasks  -  Modem  information  systems 
typically  access  and  present  a  very  large  quantity  of  data  and  present  information  in  a  variety  of  formats  from 
digital  to  advanced  graphic  formats.  Graphic  display  formats  can  represent  information  related  to  plant  functions, 
processes,  and  systems  in  ways  that  are  different  from  conventional  CR  instrumentation  and  earlier  computer- 
based  display  systems.  For  example,  graphic  displays  contain  many  more  elements  (e.g.,  abbreviations,  labels, 
icons,  symbols,  coding,  and  highlighting)  than  conventional  ones.  As  displays  convey  more  information  at 
multiple  levels  of  abstraction,  the  complexity  of  the  elements  becomes  greater.  The  cognitive  principles  that 
support  the  effectiveness  of  these  formats,  and  their  relationship  to  operators’  tasks,  are  becoming  better 
understood.  A  greater  understanding  is  emerging  about  the  effectiveness  of  designs  of  display  pages,  including 
how  display  formats  are  combined  to  produce  effective  pages,  how  different  formats  might  interact  within  a  display 
page,  and  how  complex  information  should  be  divided  to  form  individual  pages.  Thus,  guidelines  on  human 
factors  that  were  developed  for  simple  displays  may  not  be  appropriate  for  the  complex  ones  that  are  being 
developed  for  computer-based  display  systems.  If  properly  implemented,  graphical  displays  can  enhance  detection, 
recognition,  and  comprehension  of  important  information.  However,  improper  implementation  may  impair  an 
operator’s  performance.  For  example,  visual  clutter  may  be  increased,  which  may  inhibit  an  operator  from 
detecting  important  information.  New  display  formats  may  increase  mental  workload  associated  with  recognizing 
and  comprehending  information.  Inefficient  distribution  of  information  among  display  pages  may  result  in  high 
demands  on  an  operator’s  short-term  memory  (e.g.,  remembering  values  from  previous  displays)  or  a  loss  of 
understanding  of  plant  status. 

Limited  Display  Area  -  A  limited  display  area  in  visual  display  unit  (VDU)-based  systems  has  been  found  to  be  a 
problem.  Further,  compact  control  consoles  and  computer-based  display  devices,  which  contain  multiple  levels  of 
information,  may  not  be  consistent  with  the  scanning  and  recognition  strategies  of  operators  (NAS,  1995).  When . 
the  information  needed  by  an  operator  is  contained  on  more  than  one  display  screen  and  only  one  screen  can  be 
displayed  at  one  time,  the  operator  may  be  required  to  make  rapid  transitions  between  screens,  try  to  remember 
values,  or  write  them  on  paper.  Compared  to  the  sweeping  wall-to-wall  panels  of  older  conventional  CRs,  CRs  that 
use  these  computer-based  displays  provide  a  more  restricted  view,  even  when  multiple  display  devices  are 
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provided.  Further,  since  operators  can  only  monitor  what  they  can  see,  they  may  concentrate  excessively  on 
selected  areas  while  ignoring  others. 

Lack  of  Overview  -  A  vast  amount  of  information  may  be  available  at  different  levels  of  abstraction  (e.g., 
functional  versus  physical  characteristics  of  the  plant)  and  level  of  detail.  This  information  may  be  distributed 
among  separate  display  pages.  As  a  result  of  the  quality,  ty  pe,  and  distribution  of  information  in  the  display 
system,  operators  may  experience  difficulty  in  obtaining  a  rapid,  overall  assessment  of  the  plant’s  status.  The  lack 
of  an  adequate  overall  assessment  may  impair  the  operator’s  situation  awareness  and  ability  to  respond  to 
transients  and  accidents. 

Improper  Integration  and  Presentation  of  Numerical  Data  -  Digital  displays  can  summarize  and  combine 
numerical  data  from  a  wide  range  of  sources  and  present  them  in  new  ways,  such  as  calculating  simple,  weighted, 
and  running  time-interval  averages;  summed  values;  and  differences  between  values.  If  properly  implemented, 
these  displays  can  support  an  operator’s  understanding  of  the  plant.  However,  the  improper  presentation  of 
integrated  data  from  the  plant  may  confuse  or  mislead  operators.  Operators  may  not  fully  understand  the  nature  of 
the  data  that  are  contained  in  the  display,  the  data  used  in  calculations,  or  the  calculations.  These  factors  may  also 
interfere  with  the  operator’s  ability  to  verify  displayed  values  against  other  data  from  the  plant.  For  example,  a 
digital  SPDS  display  may  present  the  average  of  a  set  of  redundant  wide-range  measurement  instruments,  rather 
than  presenting  the  individual  values.  If  the  average  is  carried  out  to  one  or  more  decimal  places,  it  may  appear  to 
have  the  accuracy  of  a  narrow-range  measurement  instrument  and,  thus,  may  mislead  the  operator.  As  another 
example,  a  polar-graphic  display  (e.g.,  octagon)  may  plot  the  current  value  of  a  variable  against  a  reference  value. 
However,  the  basis  for  determining  the  reference  value  may  not  be  apparent  to  the  operator. 

Once  the  trends  in  HS1  design  and  the  potential  human  performance  issues  associated  with  them  were  identified, 
each  trend  (O’Hara,  Stubler,  and  Higgins,  1996),  was  evaluated  for  its  potential  safety  significance.  The 
evaluation  indicated  that  display  design  and  organization  was  potentially  safety  significant  and  that  further 
research  was  needed  (Stubler,  Higgins,  and  O’Hara,  1996).  That  research  is  the  subject  of  this  report. 

1.2  Scope  of  the  Research 

The  literature  addressing  the  design  of  information  systems  is  extensive.  Reviewing  all  of  it  was  well  beyond  the 
scope  of  the  project.  Therefore,  we  focused  our  efforts  on  those  aspects  of  information  system  design  that  are  most 
relevant  to  the  nuclear  industry,  where  the  research  is  most  active,  and  where  the  results  are  reflected  in  actual 
NPP  design  efforts:  information  requirements,  representation,  and  organization.  Information  requirements  refer 
to  what  information  operators  need  to  safely  and  efficiently  operate  the  plant.  This  is  a  vital  step  in  design  of 
information  systems  and  recent  approaches  to  its  analysis,  such  as  ecological  interface  design  approaches  that 
largely  stem  from  the  nuclear  and  process  control  industries,  promise  to  enhance  the  analysis  of  information  needs 
(e.g.,  Vicente  and  Rasmussen,  1992). 

Information  representation  refers  to  how  the  needed  information  is  displayed  in  the  control  room.  The 
development  of  novel  graphical  means  to  represent  information  in  a  format  that  supports  immediate  and  accurate 
understanding  of  plant  conditions  has  a  long  history  in  the  nuclear  industry  stemming  from  the  use  of  simple 
polargraphic  displays  for  safety  information  (e.g.,  Woods,  Wise,  and  Hanes,  1981)  to  more  recent  efforts  to 
represent  the  thermodynamic  state  of  the  plant  using  graphic  displays  based  on  the  Rankine  cycle  (Beltracchi, 
1995). 

Finally,  structure  is  imposed  on  the  information  representations  by  organizing  them  into  display  pages  and  the 
pages  into  an  entire  display  network.  This  is  a  significant  issue  in  process  control  facilities,  because  the  networks 
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may  contain  hundreds  or  even  thousands  of  display  pages  (O’Hara,  Stubler,  and  Higgins,  1996).  While  this  aspect 
of  information  system  design  has  been  generally  neglected,  recent  research  suggests  that  organization  is  significant 
to  operator  performance,  and  some  common  organizational  schemes  can  make  it  difficult  for  operators  to  perform 
their  tasks  (Heslinga  and  Herbert,  1995). 

The  specific  information  determined  to  be  needed  by  operators,  its  representations,  and  its  organization  into  an 
entire  information  network  can  significantly  impact  operator  performance  and  nuclear  safety. 

1.3  Organization  of  the  Report 

The  report  is  divided  into  two  parts.  Part  1  describes  the  guidance  development  methodology  and  basis.  The 
objectives  of  the  study  are  described  in  Section  2,  and  the  guidance  development  methodology  is  described  in 
Section  3.  Section  4  presents  a  characterization  of  information  systems.  Section  5  discusses  the  literature  and 
information  that  served  as  the  technical  basis  upon  which  the  review  guidance  was  developed.  The  actual  use  of 
the  technical  information  to  develop  guidance  is  discussed  in  Section  6.  Section  7  summarizes  the  guidance 
development.  References  are  in  Section  8. 

Part  2  contains  the  results  of  the  process  for  formulating  the  guidance,  and  is  presented  in  tw  o  sections.  Section  9 
discusses  the  design  process  considerations  for  reviewing  information  systems,  and  Section  10  contains  the  HFE 
design  guidelines  for  the  review  of  an  implemented  information  system  design. 
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The  objective  of  this  study  was  to  develop  HFE  review  guidance  for  information  systems  -  specifically  with  respect 
to  information  requirements  analysis,  information  representation,  and  information  organization  -  based  on  a 
technically  valid  guidance  development  methodology.  To  support  this  objective,  the  following  tasks  were 
undertaken: 

•  Development  of  a  characterization  framework  for  describing  key  design  characteristics  of  information  systems 

•  Development  of  a  technical  basis  using  research  and  analyses  of  human  performance  that  are  relevant  to 
information  systems 

•  Development  of  HFE  review  guidelines  for  information  systems  in  a  format  that  is  consistent  with  NUREG- 
0700,  Rev.  1,  and  other  NRC  review  guidance 

•  Identification  of  remaining  information  system  issues  for  which  research  results  were  insufficient  to  support 
developing  NRC  review  guidance 
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3.1  Overview 


Figure  3.1  shows  the  methodology  for  the  overall  guidance  development  for  NUREG-0700.  The  process  for  this 
development  is  discussed  in  detail  elsewhere  (O’Hara,  Brown  and  Nasta,  1996).  The  methodology  that  applies  to 
this  report  and  project  is  boxed  in  the  figure.  This  section  describes  the  rationale  for  general  guidance 
development. 


Figure  3.1  Major  Steps  in  Developing  NUREG-0700  Guidance 


The  methodology  was  guided  by  the  following  objectives: 

•  Establish  a  process  that  will  result  in  valid,  technically  defensible,  review  criteria. 

•  Establish  a  generalizable  process  that  can  be  applied  to  any  aspect  of  HSI  technology  for  w'hich  review 
guidance  is  needed. 

•  Establish  a  process  that  optimally  uses  available  resources,  i.e.,  develop  a  cost-effective  methodology. 

The  methodology  places  a  high  priority  on  establishing  the  validity  of  the  guidelines.  Validity  is  defined  along  two 
dimensions:  internal  and  external  validity.  Internal  validity  is  the  degree  to  which  the  individual  guidelines  are 
based  on  an  auditable  technical  basis.  The  technical  basis  is  the  information  upon  which  the  guideline  is 
established  and  justified.  The  technical  bases  vary  for  individual  guidelines.  Some  guidelines  may  be  based  on 
technical  conclusions  from  a  study  of  empirical  research,  some  on  a  consensus  of  existing  standards,  while  others 
are  based  on  judgement  that  a  guideline  represents  good  practices  based  on  the  information  reviewed.  Maintaining 
an  audit  trail  from  each  guideline  to  its  technical  basis  serves  several  purposes  by  enabling  the  following: 

•  Technical  merit  of  the  guideline  to  be  evaluated  by  others 

•  A  more  informed  application  of  the  guideline  since  its  basis  is  available  to  users 

•  Deviations  or  exceptions  to  the  guideline  to  be  evaluated 
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External  validity  is  the  degree  to  which  the  guidelines  are  independently  peer  reviewed.  Peer  review  is  a  good 
method  of  screening  guidelines  for  conformance  to  accepted  HFE  practices  and  for  comparing  guidelines  to  the 
practical  operational  experience  of  HSIs  in  real  systems. 

For  individual  guidelines,  these  forms  of  validity  can  be  inherited  from  the  source  documents  that  form  their 
technical  basis.  Some  HFE  standards  and  guidance  documents,  for  example,  already  have  good  internal  and 
external  validity.  If  validity  is  not  inherited,  however,  it  should  be  established  as  part  of  the  process  for  guidance 
development.  Methodology  was  established  to  provide  validity  both  inherited  from  its  technical  basis  and  through 
developing  and  evaluating  guidance. 

Figure  3.2  shows  the  process  used  to  develop  the  technical  basis  and  guidance.  The  process  emphasizes 
information  sources  that  have  the  highest  degree  of  internal  and  external  validity  for  developing  the  technical 
basis.  Thus,  primary  and  secondary'  source  documents  were  sought  as  sources  of  guidance  first,  followed  by 
tertiary  source  documents,  basic  literature,  industry  experience,  and  other  sources.  From  these,  we  identified 
design  principles  and  lessons  from  industry  experience.  Using  this  technical  basis  as  a  foundation,  the  guidance 
was  developed.  For  specific  aspects  of  the  topic,  in  which  the  technical  basis  was  inadequate  for  developing 
guidance,  we  defined  unresolved  research  issues.  Thus,  the  technical  basis  led  to  the  development  of  both  guidance 
and  issues.  The  resulting  guidance  documentation  includes  HFE  guidelines,  technical  basis,  the  development 
methodology,  and  unresolved  research  issues. 

Each  of  the  steps  of  this  research  -  topic  characterization,  development  of  technical  basis,  guidance  development 
and  documentation,  identification  of  issues,  and  peer  review  -  is  discussed  in  greater  detail  in  the  sections  that 
follow. 


3.2  Characterization  of  the  Information  System 

The  first  step  in  developing  guidance  was  to  identify  the  areas  for  which  it  was  needed.  We  reviewed  existing 
information  systems  to  identify  the  dimensions  and  characteristics  along  which  information  systems  can  be 
defined.  The  characterization  was  important  because  it  provided  a  structure  within  which  the  reviewer  could 
request  information  about  a  system  and  with  which  to  structure  the  guidance  for  design  review.  The 
characterization  of  information  systems  is  presented  in  Section  4  . 

3.3  Development  of  the  Technical  Basis 

We  began  to  formulate  detailed  review  guidelines  by  collecting  technical  information  on  which  guidance  would  be 
based  (Figure  3.2).  The  process  was  designed  to  formulate  valid  guidance  cost  effectively.  First,  primary  source 
documents,  HFE  standards  and  guidance  documents  possessing  internal  and  external  validity,  were  sought.  That 
is,  the  primary  source  documents  generally  provided  their  own  research  bases,  and  the  document  developers 
considered  the  available  research  and  operational  experience.  Using  their  knowledge  and  expertise,  they  developed 
HFE  guidelines.  These  primary  source  documents  were  often  extensively  peer  reviewed.  The  documents  added 
tremendous  value  to  individual  research  reports.  They  were  developed  by  experts  who  consider  research  in  terms 
of  its  applicability  and  generalizability  to  real  systems,  include  knowledge  and  expertise  gained  through 
operational  experience  and  guidance  application,  and  modify  the  guidance  based  on  extensive  peer  review. 

However,  we  found  little  information  that  addressed  advanced  display  design.  While  several  documents  address 
display  design  in  nuclear  plants,  their  applicability  and  guidance  for  advanced  display  systems  is  limited.  These 
documents  are 
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Figure  3.2  Process  for  Developing  Technical  Basis  and  Guidance 

Electric  Power  Research  Institute’s  Advanced  Light  Water  Reactor  Utility  Requirements  Document  (URD) 
(EPR1,  1993) 

International  Electrotechnical  Commission’s  (IEC)  Design  for  Control  Rooms  of  Nuclear  Power  Plants  (IEC- 
964)  (IEC,  1989) 
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•  Institute  of  Electrical  and  Electronics  Engineers'  (IEEE)  Guide  for  the  Application  of  Human  Factors 
Engineering  in  the  Design  of  Computer-Based  Monitoring  and  Control  Displays  for  Nuclear  Generating 
Stations  (Standard  PI 289)  (IEEE,  1996) 

Section  10.4.4,  Displays,  of  the  EPRJ  URD  addresses  display  requirements  in  advanced  NPPs.  Requirements  are 
identified  for  trend  displays,  integrating  displays  and  mimics  (similar  to  group  view  displays  discussed  in  Stubler 
and  O’Hara,  1996),  and  electronic  displays  at  workstations.  However,  the  URD  does  not  address  specific  formats 
or  how  displays  should  be  organized.  It  states  that  the  detailed  design  of  displays  should  be  tested  using 
simulation. 

Section  4.6,  Information  Systems,  of  IEC-964  discusses  displays  in  general  and  refers  to  Appendix  A  Section 
A.4.6,  Information  Systems,  for  more  details.  However,  neither  section  gives  more  than  high-level  principles,  such 
as  the  need  for  consistency,  and  they  do  not  address  specific  requirements  for  individual  display  formats,  or  the 
organization  of  information  within  the  network. 

IEC-964  also  contains  a  supplement  on  "Visual  Display  Unit  Application  to  Main  Control  Room  in  Nuclear  Power 
Plants."  The  supplement  gives  applications,  advantages,  and  disadvantages  of  alphanumerical  displays,  bar 
graphs,  trend  curves,  system  status  displays,  and  logic  displays.  However,  there  is  no  detailed  guidance  for  any 
formats,  and  more  advanced  displays  and  organization  are  not  addressed.  IEC-964,  like  the  URD,  requires 
simulator  testing  as  part  of  the  verification  and  validation  (V&V)  for  advanced  VDU-based  sy  stems. 

Section  6,  Typical  Display  Format  Types,  of  IEEE-PI289  addresses  graphs,  mimics,  logic  displays,  faceplate 
displays,  alphanumeric  displays,  and  integrated  displays  and  discusses  the  application,  advantages  and 
disadvantages  of  each.  The  integrated  display  refers  to  a  display  that  integrates  several  types  of  displays  within  a 
single  one  and  should  not  be  confused  with  an  integral  display. 

Thus,  the  scope  of  these  documents  is  addressed  in  NUREG-0700,  Rev.  1  (Section  5.1  addresses  the  scope  of 
NUREG-0700  guidance),  and  therefore,  guidance  for  more  advanced  information  systems,  the  focus  of  our  current 
research,  is  not  addressed.  Consequently,  while  developing  a  technical  basis  for  information  systems  we 
considered  the  other  sources  shown  in  Figure  3.2.  Secondary  sources  were  documents  for  which  either  internal  or 
external  validity  was  established.  Documents  for  which  neither  type  of  validity  was  established  were  considered 
tertiary  sources.  The  preference  was  to  use  documents  for  which  validity  had  been  established. 

This  information  includes  the  results  from  basic  literature  that  we  analyzed.  While  the  information  pertaining  to 
advanced  information  systems  was  limited  in  current  standards  and  guidance  documents,  the  situation  is 
completely  reversed  in  the  basic  literature.  The  literature  is  extensive;  therefore,  we  focused  on  using  published 
literature  reviews  where  available,  and  supplementing  them  with  more  recent  published  accounts  of  individual 
studies. 

When  guidance  was  based  on  basic  literature,  engineering  judgement  was  required  to  generalize  from  the  unique 
aspects  of  individual  experiments  and  studies  to  actual  applications  in  the  workplace.  This  is  because  individual 
experiments  have  unique  constraints  that  limit  their  generalizability  (such  as  their  unique  participants,  types  of 
tasks  performed,  and  types  of  equipment  used).  For  example,  laboratory  experiments  often  do  not  involve  tasks  of 
the  complexity  of  NPP  operations,  and  most  experiments  do  not  examine  tasks  under  the  same  performance 
shaping  factors  (such  as  rotating  shifts,  stress,  and  fatigue)  that  exist  in  a  work  environment.  While  information 
from  research  is  a  valuable  part  of  developing  guidance,  it  usually  cannot  be  blindly  adopted.  Thus,  the  results 
must  be  interpreted  in  the  context  of  real-world  tasks  and  systems,  which  involves  judgement  based  on  professional 
and  operational  experience. 
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Industry  experience  also  was  used,  such  as  published  case  studies  and  surveys  and  interviews  with  knowledgeable 
domain  experts.  Site  visits  and  interviews  with  operators,  trainers,  and  designers  were  conducted  as  part  of  an 
earlier  phase  of  this  project,  and  pertinent  information  from  that  effort  was  used.  Although  such  information  may 
lack  a  rigorous  experimental  basis  (and  thus,  a  measure  of  validity),  it  is  highly  relevant. 

Finally,  some  information  was  identified  in  original  research.  A  full  account  of  the  research  is  published  elsewhere 
(Roth  and  O’Hara,  1998).  Original  research  has  the  advantage  of  enabling  a  study  to  be  focused  on  the  specific 
issues  that  need  to  be  addressed  in  guidance  development.  However,  because  of  the  time  and  resources  required  to 
conduct  original  research,  it  is  only  used  when  important  information  is  needed  that  cannot  be  obtained  through 
other  means. 

Empirical  evaluations  of  information  systems  and  characteristics  provided  a  substantial  portion  of  the  technical 
basis  upon  which  to  develop  design  review  guidance.  The  general  criteria  by  which  such  studies  were  evaluated 
will  be  briefly  described.  One  of  the  most  important  considerations  is  that  the  information  system  studies  provided 
a  basis  from  which  conclusions  can  be  generalized  beyond  the  specific  individual  study.  To  make  this  evaluation, 
the  studies  were  considered  within  the  context  of  validation  studies  where  generalization  (external  validity)  is  a 
primary  consideration. 

O’Hara  et  al.  (1997)  discussed  the  detailed  methodological  considerations  for  validation  of  complex  human- 
machine  systems.  A  conceptual  approach  to  validation  was  developed  that  identified  important  validation 
principles  and  their  relationships.  The  general  concepts  are  concerned  with  (1)  establishing  the  requirements  for 
making  a  logical  and  defensible  inference  from  validation  tests  to  predicted  integrated  system  performance  under 
actual  operating  conditions,  and  (2)  identifying  the  aspects  of  validation  methodology'  that  are  important  to  the 
inference  process.  The  technical  basis  for  inference  in  validation  is  based  upon  four  general  forms  of  validity: 
system  representation,  performance  representation,  test  design,  and  statistical  conclusion. 

System  representation  validity  refers  to  the  degree  to  which  the  tests  include  those  aspects  of  the  system  that  are 
important  to  real-world  conditions.  Specifically,  this  validity  is  based  on  the  representativeness  of  the  system 
model,  human-system  interface,  personnel,  and  operational  events.  The  inference  process  is  supported  to  the 
extent  that  important  aspects  of  the  integrated  system  are  represented  with  high  fidelity,  and  to  the  extent  to  which 
important  contributors  to  potential  variability  in  system  performance  have  been  adequately  sampled. 

Performance  representation  validity  refers  to  the  completeness  and  representativeness  of  the  performance  measures. 
A  comprehensive,  hierarchal  approach  to  evaluation  guided  by  supervisory  control  theory  may  be  used  to  specify 
important  aspects  of  performance  ranging  from  operator  cognitive  processes  to  system  functions.  In  general,  the 
effects  can  be  related  to  (1)  personnel  role  -  a  change  in  functions  and  responsibilities  of  plant  personnel; 

(2) primary  tasks  -  a  change  in  the  way  that  personnel  perform  their  primary  tasks,  such  as  process  monitoring, 
situation  assessment,  response  planning,  and  response  implementation  and  control;  (3)  secondary  tasks  -  a  change 
in  the  tasks  the  operator  must  perform  when  interacting  with  the  information  system,  such  as  navigating  through 
displays  and  searching  for  data;  (4)  cognitive  factors  -  such  as  a  change  in  cognitive  workload;  and  (5)  personnel 
factors  -  a  change  in  the  required  qualifications  or  training  of  plant  personnel.  Performance  measures  must  be 
used  that  can  address  these  effects.  Failure  to  include  measures  of  all  important  performance  variables,  poor 
measurement  properties,  and  poorly  specified  criteria  weaken  this  validity  and  the  ability  to  generalize  the  results. 

Test  design  validity  addresses  the  experimental  design  and  test  procedures.  Experimental  design  considerations 
address  the  suitability  of  the  definitions  of  independent  variables.  Inappropriate  test  procedures  can  bias  the 
relationship  between  the  observations  of  performance  and  the  integrated  system,  and  thus  undermine  their  causal 
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linkage.  When  factors  introduced  by  the  test  methodology  weaken  the  ability  to  interpret  the  system-performance 
correlation,  validity  is  compromised. 

Finally,  statistical  conclusion  validity  addresses  the  relationship  between  the  observed  data  and  established  criteria 
of  performance. 

While  the  studies  reviewed  in  Section  5  were  evaluated  for  these  types  of  validity  and  their  associated 
methodological  considerations,  such  an  analysis  is  limited  by  the  extent  to  which  the  studies  are  documented. 

3.4  Development  of  Guidelines  and  Documentation 

Once  the  technical  information  was  assembled,  a  draft  set  of  guidelines,  which  are  in  Part  2,  was  developed  from 
the  information  in  the  technical  basis.  The  guidelines  were  organized  and  specified  in  a  standard  format  (which  is 
discussed  in  Section  6). 

3.5  Identification  of  Issues 

Where  there  was  insufficient  information  to  provide  a  technical  basis  upon  which  to  develop  valid  design  review 
guidance,  an  issue  was  defined.  Section  7  describes  these  issues. 

From  a  research  standpoint,  issues  reflect  aspects  of  the  design  and  use  of  information  systems  that  will  require 
additional  investigation  to  resolve.  From  a  design  review  standpoint,  issues  reflect  aspects  of  design  and  use  that 
will  have  to  be  addressed  on  a  case-by-case  basis.  For  example,  an  issue  can  be  addressed  as  part  of  design-specific 
tests  and  evaluations. 

3.6  Peer  Review 

The  resulting  technical  basis  and  guidance  was  submitted  for  review  by  individuals  with  knowledge  and  expertise 
related  to  the  review  topic.  Included  were  reviews  by  personnel  from  the  U.S.  NRC  with  expertise  in  HFE  and 
engineering  fields  directly  related  to  the  topic.  Human  factors  specialists  who  are  external  to  the  NRC  and  have 
expertise  in  human  performance  in  complex  systems,  such  as  NPPs  and  aviation,  conducted  additional  reviews. 
These  external  reviews  include  evaluations  of  the  topic  characterization  along  the  criteria  of  clarity,  accuracy,  and 
completeness,  and  the  technical  basis  along  the  criteria  of  organization,  necessity,  sufficiency,  resolution,  and 
basis.  Comments  from  the  peer  reviews  were  incorporated  into  the  current  version  of  this  document. 
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The  plant  information  system  refers  to  those  aspects  of  the  HS1  that  give  information  to  the  operator  about  the 
plant  state.  NUREG-0700,  Rev.  1  provided  a  preliminary  characterization  of  information  system  components.  In 
this  section,  that  characterization  is  made  more  definitive  and  expanded  to  accommodate  the  results  of  this 
research. 

An  information  system  has  the  following  major  components:  information  requirements,  representation  system, 
management  functions,  and  display  devices.  Each  is  briefly  described  below.  This  section  gives  a  general 
characterization  of  information  systems,  whether  they  are  advanced  or  not.  The  term  advanced,  as  used  in  this 
report,  refers  to  information  systems  that  use  novel  graphic  forms,  such  as  a  Rankine  Cycle  display,  and  go  beyond 
conventional  forms,  such  as  bar  charts  and  trend  graphs.  This  general  characterization  is  consistent  with  other 
approaches  to  define  the  hierarchical  organization  of  information  and  display  systems  (e.g.,  Woods  et  al.,  1991). 

This  report  addresses  the  first  two  aspects  of  the  characterization:  information  requirements  and  representation 
systems.  The  topic  of  management  functions  is  being  addressed  in  another  NRC  research  project  because  of  its 
general  applicability  to  human-computer  interfaces,  such  as  soft  controls  and  computer-based  procedures,  in 
addition  to  information  systems.  The  topic  of  display  devices  is  hardware  related  and  is  being  considered 
separately. 

4.1  Information  Requirements 

Information  requirements  refer  to  the  information  operators  need  to  monitor  and  control  the  plant.  Information  is 
at  the  center  of  human  functions  in  any  complex  system.  The  operators’  information  needs  are  the  requirements 
that  must  be  addressed  in  the  design  of  the  information  system.  The  determination  of  what  information  is  needed 
is  referred  to  as  information  requirements  analysis.  New  approaches  to  information  analyses  conducted  to  define 
the  information  requirements  of  display  systems  have  emerged  and  will  be  discussed  in  Section  5.3. 

4.2  Representation  System 

While  the  information  requirements  identify  what  information  operators’  need,  the  way  in  which  that  information 
is  presented  to  operators  is  called  the  representation  system.  Information  is  given  to  the  operators  via  some  sensory 
channel.  In  a  nuclear  plant,  it  is  predominantly  the  operator’s  visual  and  auditory  channels  that  are  used  to  convey 
information  about  the  plant.  The  representation  system  is  the  means  by  which  information  is  communicated  to 
operators;  it  contains  the  following  components:  elements,  formats,  pages,  and  networks.  Each  is  discussed  below. 
Figure  4.1  shows  the  representation  system  (along  with  management  functions  and  display  devices). 

Display  Elements 

Display  elements  refer  to  the  basic  building  blocks  that  are  used  to  make  up  display  formats,  such  as  abbreviations, 
labels,  icons,  symbols,  coding,  and  highlighting. 

Display  Format 

Display  formats  refer  to  the  general  class  of  information  presentation.  Examples  of  general  classes  are  continuous 
text  (such  as  a  procedure  display),  mimics  and  piping  and  instrumentation  diagram  (P&ID)  displays,  trend  graphs, 
and  flowcharts.  This  aspect  of  representation  systems  has  been  most  affected  by  recent  trends  in  design  of 
information  systems.  The  ability  of  computer  graphics  to  portray  an  essentially  limitless  set  of  novel  graphic  forms 
has  offered  great  possibilities  to  provide  operators  with  enhanced  representations  of  the  plant.  Woods  et  al.  (1991) 
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Figure  4.1  Display  System  Characterization 
(Information  requirements  not  shown.) 


refer  to  this  level  as  "Graphic  Forms"  and  consider  it  the  "fundamental  hinge  in  the  series  of  levels."  The  same  can 
be  said  of  display  formats  in  the  current  characterization.  Below  this  level,  the  characterization  seeks  to  define  the 
building  blocks  of  display  formats.  Above  this  level,  formats  are  successively  organized  into  pages  and  then  into 
networks.  Thus,  display  formats,  which  are  discussed  in  Section  5.4,  are  the  most  significant  "unit  of  analysis"  of 
the  information  system. 

There  is  a  grammatical  aspect  of  display  formats  similar  to  human  language.  In  both  cases  the  grammar  describes 
the  relationship  between  the  physical  form  (e.g.,  sentence  or  visual  aspects  of  a  graphic  display)  and  its  meaning. 
The  link  between  physical  form  and  meaning  is  made  using  transformation  rules.  To  illustrate  this  point,  an 
example  from  linguistics  is  described,  followed  by  an  example  from  graphic  displays.  Figure  4.2  shows  the  link 
between  a  sentence  and  its  meaning  through  transformational  rules.  Panel  (a)  in  Figure  4.2  shows  the  case  where 
the  physical  form  (sentence)  is  unambiguously  linked  to  meaning  (i.e.,  the  sentence  form  has  only  one  grammatical 
meaning).  However,  some  sentence  structures  are  ambiguous;  i.e.,  the  same  physical  form  (sentence)  can  lead  to 
more  than  one  meaning  (see  panel  (b)  Figure  4.2).  The  left  sentence  is  about  apple  eaters,  while  the  right  describes 
a  type  of  apple  that  is  suitable  for  eating. 

Just  as  the  same  sentence  can  have  two  distinct  meanings,  two  very  different  sentence  structures  can  have  the  same 
meaning.  Thus,  there  is  not  always  a  one-to-one  mapping  between  the  physical  form  and  semantics. 

Similar  relationships  can  exist  between  a  graphic  display  and  its  meaning(s).  "Display  semantics"  pertains  to  the 
meaning  of  the  display  form  for  the  plant  (see  Figure  4.3).  The  semantics  must  consider  the  instrumentation  and 
the  data  processing  that  are  used  to  drive  its  format.  Thus,  the  display’s  grammar  describes  the  correspondence 
between  the  display  and  the  aspects  of  the  system  it  represents  by  relating  its  physical  form  and  functions  to  its 
meaning  to  the  plant’s  functions  and  states.  There  are  syntactical  rules  that  relate  the  display  form  and  its 
semantics. 
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(b)  Ambiguous  Sentence 

Figure  4.2  The  Relationship  Between  Surface  Structure  and  Sentence  Meaning 
(Adapted  from  Reynolds  and  Flagg,  1977) 
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Figure  4.3  The  Relationship  Between  Surface  Structure  and  Semantics  in  Display  from  a 
Loss  of  Coolant  Accident  (LOCA)  in  a  Pressurized  Water  Reactor  (PWR) 

(Source:  Reprinted  with  permission  from  Woods,  D.,  Wise,  J.,  and  Hanes,  L.,  “An  Evaluation  of  Nuclear 
Power  Plant  Safety  Parameter  Display  Systems,”  Proceedings  of  the  Human  Factors  Society  25th  Annual 
Meeting.  Copyright  1981  by  the  Human  Factors  and  Ergonomics  Society.  All  rights  reserved.) 
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In  the  display’s  design,  it  is  important  to  have  a  unique  meaning  for  each  form.  Just  as  the  surface  structure  of 
language  can  be  ambiguous  with  regard  to  its  semantics,  so  can  a  display’s  form.  If  a  display  can  lead  to  more 
than  one  interpretation,  it  is  ambiguous  and  can  be  more  easily  misunderstood.  The  meaning  of  graphic  format  in 
Figure  4.3  (from  Woods  et  al.,  1981)  shows  this.  Figure  4.3  (a)  is  an  example  of  a  polargraphic,  iconic  display 
developed  to  provide  a  NPP  operator  with  a  high-level  overview  of  the  status  of  plant  safety  functions.  The  display 
is  driven  by  eight  parameters  (that  correlate  to  the  plant  safety  functions).  Each  parameter  is  represented  by  a 
spoke  on  the  format.  The  full  scale  of  the  parameters’  values  are  displayed  over  the  length  of  the  spoke,  between 
the  two  tick  marks.  Points  representing  normal  conditions  are  plotted  as  fixed  points  equidistant  from  the  center  of 
the  figure.  Connecting  the  points  results  in  a  regular  geometric  pattern,  an  octagon.  The  reference  points  are 
dynamically  scaled,  so  that  the  reference  octagon  remains  regular.  Parameters  that  are  higher  than  normal  are 
plotted  further  away  from  the  center,  and  those  that  are  lower  are  plotted  closer  to  the  center. 

When  a  disturbance  occurs,  the  individual  parameter  values  being  monitored  as  representative  of  safety  functions 
change.  This  changes  the  points  on  the  spokes,  and  hence,  a  new  "distorted"  polygon  is  drawn.  Figure  4.3  (b) 
shows  a  case  where  the  pressurizer  level  decreased  from  the  normal  value  of  60%  to  40%.  However,  this  situation 
can  have  more  than  one  interpretation.  For  example,  this  decrease  may  be  due  to  a  leak  and  subsequent  loss  of 
water  from  the  primary  system,  a  failure  of  the  normal  pressurizer  level  control  system,  or  a  failure  of  the 
pressurizer  level  instrumentation.  Thus,  the  same  physical  form  can  lead  to  distinctly  different  meanings. 

This  type  of  grammatical  relationship  does  not  only  apply  to  graphic  displays,  although  the  rules  for  graphic 
displays  may  generally  be  more  complicated.  One  of  the  lessons  from  the  Three  Mile  Island  (TMI)  accident  was 
that  the  display’s  grammar  can  be  significant  even  for  simple  displays.  The  control  board  at  TMI  contained  a  light 
indicating  the  position  of  a  relief  valve  on  the  pressurizer.  Operators  thought  that  the  valve  was  physically  open 
when  the  light  was  on  and  closed  when  it  was  off.  However,  the  true  meaning  of  the  display  was  that  the  valve 
solenoid  operator  was  either  energized  or  not.  During  the  accident,  the  relief  valve  was  actually  stuck  open  and  not 
closed  as  the  operators  thought,  even  though  the  indicator  light  had  gone  out.  There  was  a  disconnect  between  the 
display  form,  the  status  indicator,  and  its  meaning  (solenoid  energized)  that  contributed  to  a  significant 
misunderstanding  about  the  plant. 

A  polargraphic  display  similar  to  that  shown  in  Figure  4.3  also  can  be  useful  for  analysis  during  major  transients 
and  accidents.  In  this  case,  the  semantics  of  the  display  are  more  involved.  Figure  4.4  shows  the  figure  resulting 
from  a  significant  loss  of  coolant  accident  (LOCA)  in  a  pressurized  water  reactor  (PWR)  In  a  LOCA,  a  major 
pipe  break  occurred  in  the  primary  system,  resulting  in  a  large  loss  of  radioactive  reactor  coolant  to  the 
containment’s  atmosphere.  In  Figure  4.4,  pressurizer  (PRZR)  level,  reactor  vessel  (RV)  level,  and  reactor  coolant 
system  (RCS)  pressure  decreased  due  to  the  water  leak.  Core  exit  temperature  increased  from  the  loss  of  water  and 
resultant  lack  of  cooling  to  the  reactor  fuel.  Containment  pressure  also  increased  due  to  the  high  pressure/high 
temperature  steam  and  water  mix  flowing  into  the  containment  from  the  RCS.  The  radiation  level  in  the 
containment  also  increased  due  to  radioactive  water  and  steam  in  containment.  As  shown  in  Figure  4.3  (b),  as  the 
values  of  the  individual  parameter  change,  the  polygon  distorts.  For  the  LOCA  situation,  the  amount  of  distortion 
indicates  a  large  deviation  in  many  parameters  from  normal  (still  indicated  by  the  dotted  regular  octagon). 
However,  the  operator  cannot  draw  any  specific  conclusions  from  the  shape  of  the  polygon,  but  rather  must  think 
about  the  meaning  of  each  parameter’s  deviation  and  then  about  what  they  mean  in  the  aggregate.  This  requires 
substantial  training  and  is  aided  by  detailed  emergency  procedures. 

For  any  information  display  format,  it  is  important  to  clearly  understand  all  three  components  of  the  display’s 
grammar:  the  format  itself,  its  semantics  or  meaning,  and  the  rules  that  relate  the  two. 
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Figure  4.4  An  Example  of  More  Complex  Display  Semantics  from  a  Loss  of  Coolant 
Accident  (LOCA)  in  a  Pressurized  Water  Reactor  (PWR) 

(Source:  Reprinted  with  permission  from  Woods,  D.,  Wise,  J.,  and  Hanes,  L.,  “An  Evaluation  of 
Nuclear  Power  Plant  Safety  Parameter  Display  Systems,”  Proceedings  of  the  Human  Factors  Society 
25th  Annual  Meeting.  Copyright  1981  by  the  Human  Factors  and  Ergonomics  Society.  All  rights 
reserved.) 

Display  Pages 


Display  page  refers  to  a  defined  set  of  information  that  is  intended  to  be  displayed  as  a  single  unit.  Typical  display 
pages  of  NPPs  may  combine  several  different  formats  on  a  single  VDU  screen,  e.g.,  combining  bar  charts  and 
digital  displays  in  a  graphic  P&ID  format. 


What  goes  into  one  display  page,  i.e.,  the  integration  of  formats  that  make  up  the  page,  is  typically  meant  to  show 
some  aspect  of  the  process.  For  example,  one  page  may  provide  a  high-level  status  overview  of  the  primary  system. 


Display  pages  typically  have  a  label  and  designation  within  the  computer  system  so  they  can  be  accessed  by 
operators  as  a  single  "display."  Section  5.5  addresses  display  pages. 

Display  Networks 

Complex  systems,  such  as  NPPs,  are  not  represented  by  a  single  graphic  display.  In  fact,  for  many  recently  built 
plants,  the  numbers  of  display  pages  is  more  typically  in  the  hundreds  or  thousands  (O’Hara,  Stubler,  and  Higgins, 
1996).  Display  networks  refer  to  an  entire  set  of  display  pages  within  an  information  system. 

To  perform  their  functions  and  tasks,  operators  must  access  these  pages.  When  the  number  of  pages  is  large, 
knowing  where  information  is  located  is  difficult.  Thus,  the  organizational  structure  of  the  network  within  the 
information  system  is  an  important  consideration.  A  common  type  of  organizational  structure,  for  example,  is  by 
plant  system.  Section  5.5  addresses  display  networks. 
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4.3  Management  Functions 

As  discussed  above,  display  networks  can  include  hundreds  or  more  pages.  Further,  the  information  may  be  visible 
within  a  limited  display  area  provided  by  video  display  units  (VDUs).  Thus,  facilities,  such  as  navigating  within 
the  display  network  to  retrieve  displays,  to  manipulating  display  pages  (such  as  sizing  or  moving  windows),  and 
moving  within  a  single  display  page  (such  as  scrolling  and  zooming),  are  needed  so  that  operators  can  manage  the 
information.  Generically,  these  are  called  interface  management  functions.  While  they  are  important 
considerations  in  information  system  design,  they  will  not  be  addressed  in  this  report.  Interface  management  is 
being  addressed  in  a  separate  project  (O’Hara,  Stubler,  and  Nasta,  1997). 

4.4  Display  Devices 

Display  devices  are  the  media  used  to  present  information  to  plant  personnel.  Display  devices  include  meters, 
gauges,  VDUs,  and  hard-copy  display  devices  (printers  and  plotters).  NUREG-0700,  Rev.  1  contains  HFE 
guidance  for  the  characteristics  of  these  devices,  such  as  resolution  and  viewing  angle,  that  are  important  to  crew 
performance.  However,  technology  or  new  display  devices,  such  as  flat  panel  displays,  is  emerging  and  their 
characteristics  are  discussed  separately  (O’Hara,  Stubler,  and  Higgins,  1996).  Thus,  like  interface  management, 
display  devices  are  an  important  aspect  of  information  systems,  but  they  will  not  be  addressed  here. 

4.5  Relationship  to  Other  Aspects  of  HSIs 

The  relationship  between  the  design  features  and  functions  of  the  information  system  and  that  of  the  other  aspects 
of  the  HSI  should  be  addressed.  Modem  information  systems  may  be  integrated  with  other  HSI  subsystems,  such 
as  plant  controls.  For  example,  controls  may  use  on-screen  representations  that  provide  information  through 
which  operators  can  control  the  plant.  These  types  of  controls  are  referred  to  as  soft  controls.  Features  and 
functions  of  soft  control  and  their  impact  on  the  design  of  information  system  is  addressed  elsewhere  (Stubler, 
O’Hara,  and  Kramer,  2000). 

Information  systems  also  may  be  integrated  with  plant  procedure  systems,  especially  when  the  procedure  systems 
are  computer-based.  Procedure  systems  that  are  computer-based  may  be  fully  integrated  with  the  information 
system.  For  example,  data  comparisons  required  for  evaluating  procedure-step  logic  may  be  presented  using  trend 
graphs  provided  by  the  information  system  for  that  and  other  purposes  as  well.  In  addition,  the  procedure  system 
may  share  the  same  display  devices  and  interface  management  functions.  Procedure  systems  that  are  computer- 
based  and  their  impact  on  the  design  of  information  systems  also  is  addressed  elsewhere  (O’Hara,  Higgins,  Stubler, 
and  Kramer,  2000). 
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The  purpose  of  this  section  is  to  establish  a  technical  basis  upon  which  to  develop  review  guidance  for  information 
systems  and  to  identify  human  performance  issues  where  the  technical  basis  is  insufficient  for  developing 
guidance.  The  review  begins  with  a  discussion  of  the  NRC’s  current  approach  for  reviewing  the  design  of 
information  systems  (Section  5.1).  This  discussion  provides  a  baseline  against  which  new  research  can  be 
compared  to  determine  whether  new  guidance  is  necessary  based  on  recent  advances  in  these  areas. 

In  Section  5.2,  the  generic  cognitive  tasks  of  operators  are  discussed  to  define  the  general  scope  of  activities  that  an 
information  system  must  support.  Section  5.3  addresses  the  recent  approaches  to  analyzing  information 
requirements.  These  approaches  are  evaluated  for  their  research  support  as  well  as  their  contribution  to  the  NRC’s 
existing  guidance  on  design  review.  Information  representation  (Section  5.4)  and  organization  (Section  5.5)  are 
addressed  next.  Section  5.6  summarizes  the  results  and  conclusions. 

5.1  Existing  NRC  Guidance  for  Information  Systems 

The  NRC’s  approach  to  the  review  of  information  system  requirements  analysis,  representation  design,  and 
evaluation  will  be  discussed  to  form  the  basis  against  which  to  evaluate  recent  approaches  to  these  topics. 

Nuclear  plants,  like  other  complex  human-machine  systems,  are  generally  governed  by  applied  general  systems 
theor>r  (Bailey,  1982;  DeGreen,  1970;  Gagne  and  Melton,  1988;  Van  Cott  and  Kinkade,  1972;  Woodson,  1981). 
Systems  engineering  has  been  defined  as  “the  management  function  which  controls  the  total  system  development 
effort  for  the  purpose  of  achieving  an  optimum  balance  of  all  system  elements.  It  is  a  process  which  transforms  an 
operational  need  into  a  description  of  system  parameters  and  integrates  those  parameters  to  optimize  the  overall 
system  effectiveness”  (Kockler  et  al.,  1990).  Developing  a  complex  system  begins  with  the  mission  or  purpose  of 
the  system  and  the  requirements  needed  to  achieve  the  mission’s  objectives.  The  effective  integration  of  HFE 
considerations  into  the  design  is  accomplished  by  providing  (1)  a  structured  top-down  approach  to  system 
development  that  is  iterative,  integrative,  interdisciplinary,  and  requirements-driven,  and  (2)  a  management 
structure  that  details  the  HFE  considerations  in  each  step  of  the  overall  process. 

The  systems  engineering  approach  has  been  used  as  the  basis  for  many  HFE  design  approaches  developed  by  the 
NPP  industry  (EPRI,  1992;  IEC,  1989;  IEEE,  1988).  This  approach  forms  the  basis  of  the  NRC’s  review  HFE 
methodology  as  well,  e.g.,  NUREG-071 1  (O’Hara,  Higgins,  et  al.,  1994)  and  NUREG-0700,  Rev.  1.  The  key 
elements  of  this  approach  will  be  briefly  described  to  understand  the  approach  to  the  development  of  information 
systems.  (While  the  key  elements  are  essentially  the  same  across  the  different  documents,  some  of  the  specific 
details  are  slightly  different.  See  Welch  and  O’Hara  (1996)  for  a  detailed  discussion  of  systems  engineering  as  it 
relates  to  NUREG-07 1 1).  The  systems  approach  addresses  all  aspects  of  system  design;  however,  there  are  six  that 
are  important  to  developing  information  systems:  operating  experience  review,  functional  requirements  analysis, 
task  analysis,  human  reliability  analysis,  HS1  design,  and  verification  and  validation. 

Operating  Experience  Review 

The  objective  of  an  operating  experience  review  (OER)  is  to  ensure  that  HFE-related  problems  and  issues 
encountered  in  previous  designs  that  are  similar  to  the  current  design  were  identified  and  analyzed,  so  that  they  are 
avoided  in  developing  the  current  design,  or  in  the  case  of  positive  features,  to  ensure  that  they  are  retained. 

An  effective  OER  conducted  on  plants  and  systems  that  are  similar  to  one  under  design  will  identify  information 
requirements  needed  for  the  operators  to  properly  perform  their  tasks.  In  particular,  an  OER  can  identify  problem 
areas  with  current  designs  that  led  to  incidents,  and  which  can  be  addressed  at  the  design  stage. 
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Functional  Requirements  Analysis  and  Function  Allocation 

Functional  requirements  analysis  consists  of  identifying  functions  that  must  be  undertaken  to  satisfy  plant 
productivity  and  safety  objectives,  e.g.,  to  prevent  or  mitigate  the  consequences  of  postulated  accidents  that  could 
cause  undue  risk  to  the  health  and  safety  of  the  public.  Function  allocation  is  the  analysis  of  the  requirements  for 
plant  control  and  the  assignment  of  control  functions  to  personnel,  system  elements,  and  combinations  of  personnel 
and  system  elements. 

The  term  function  may  be  used  in  different  ways  by  plant  designers.  It  can  refer  to  very  high-level  plant  functions, 
such  as  critical  safety  functions,  or  the  functioning  of  an  individual  piece  of  equipment,  such  as  a  valve  or  a  wall- 
panel  information  display  system.  The  former  is  meant  here  Therefore,  a  function  is  an  action  that  is  required  to 
achieve  a  desired  goal.  A  high-level  objective,  such  as  preventing  the  release  of  radioactive  material  to  the 
environment,  is  one  that  designers  strive  to  achieve  through  implementing  other  more  specific,  high-level 
functions.  Designers  will  strive  to  achieve  these  high-level  functions  through  the  design  of  the  plant,  and  plant 
operators  will  strive  to  achieve  them  through  the  proper  operation  of  the  plant.  Examples  of  these  functions  are 
reactivity  control,  containment  integrity  and  control  of  RCS  water  mass  inventory. 

The  high-level  function  may  be  described  without  reference  to  specific  plant  systems  or  to  the  level  of  human  or 
machine  intervention  that  is  required  to  satisfy  the  function.  High-level  functions  are  usually  accomplished 
through  some  combination  of  lower-level  system  actuations,  such  as  reactor  trip,  safety  injection,  or  accumulators. 
Manipulating  lower-level  systems  to  satisfy  a  higher-level  function  is  defined  here  as  a  control  function.  Often 
plant  systems  are  used  in  combination  to  achieve  a  higher-level  function. 

The  requirements  of  these  functions  are  analyzed  to  (1)  determine  the  objectives,  performance  requirements,  and 
constraints  of  the  design;  (2)  define  the  high-level  functions  that  must  be  accomplished  to  meet  the  objectives  and 
required  performance;  (3)  define  the  relationships  between  high-level  functions  and  plant  systems  (e.g.,  plant 
configurations  or  success  paths)  responsible  for  performing  the  function;  (4)  provide  a  framework  for 
understanding  the  role  of  controllers  (whether  personnel  or  system)  for  controlling  the  plant.  Some  of  the  relevant 
NUREG-071 1  considerations  for  function  analysis  and  decomposition  include  the  following: 

•  The  functional  requirements  and  allocation  analysis  should  be  kept  current  throughout  the  development  of  the 
design  and  held  until  decommissioning  so  that  it  can  be  used  as  a  design  basis  when  modifications  are 
considered.  Control  functions  should  be  reallocated  iteratively,  in  response  to  developing  design  specifics, 
operating  experience,  and  the  outcomes  of  ongoing  analyses  and  trade-off  studies. 

•  The  functions  and  systems  should  be  described.  For  each  safety  function,  the  set  of  plant  system 
configurations  or  success  paths  that  are  responsible  for  or  capable  of  carrying  out  the  function  should  be 
clearly  defined.  Functions  should  be  diagramed  at  several  levels,  starting  at  “top-level”  functions,  where  a 
very  general  picture  of  major  functions  is  described,  and  continuing  to  lower  levels  until  a  specific  critical  end- 
item  requirement  emerges  (e.g.,  a  piece  of  equipment,  software,  or  an  operator).  The  functional  decomposition 
should  address  the  following  levels:  (1)  high-level  functions  (e.g.,  maintain  reactor  coolant  system  [RCS] 
integrity)  and  critical  safety  functions  (e.g.,  maintain  pressure  control  of  the  RCS)  and  (2)  specific  plant 
systems  and  components. 
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•  Each  high-level  function  should  be  described;  the  descriptions  should  include: 

-  Purpose  of  the  function 

Conditions  that  indicate  that  the  function  is  required 

-  Parameters  that  indicate  that  the  function  is  available 

Parameters  that  indicate  that  the  function  is  operating  (e.g.,  flow  indication) 

-  Parameters  that  indicate  that  the  function  is  achieving  its  purpose  (e.g.,  reactor  vessel  level  returning  to 
normal) 

-  Parameters  that  indicate  that  operation  of  the  function  can  or  should  be  terminated 

Some  of  the  functions  are  allocated  completely  or  in  part  to  plant  personnel.  The  functions  undergo  additional 
analyses  to  determine  human  task  requirements. 

Task  Analysis 

Task  analysis  is  the  evaluation  of  the  performance  demands  on  plant  personnel  to  identify  the  task  requirements  for 
accomplishing  the  functions  allocated  to  them  (Drury  et  al.,  1987).  It  is  a  very  important  activity  because  it  defines 
the  HSI  requirements  for  accomplishing  tasks,  including  information  requirements.  Although  there  is  no  precise 
definition  of  a  task  with  respect  to  the  level  of  abstraction,  a  task  is  a  group  of  related  activities  that  have  a 
common  objective  or  goal. 

Again,  as  in  functional  requirements  analysis,  decomposition  into  ever  increasing  levels  of  detail  is  necessary. 

Task  analyses  should  begin  on  a  gross  level  and  involve  developing  detailed  descriptions  of  what  personnel  must 
do.  The  nature  of  the  input,  process,  and  output  required  by  and  of  personnel  should  be  defined.  For  infoimation 
requirements  analysis,  the  detailed  description  of  tasks  should  include  the  follow  ing: 

•  Gathering  Information 

-  Information  required  (e.g.,  parameters,  units,  precision,  accuracy) 

Information  source  (e.g.,  alarm,  displays,  verbal  communication) 

•  Requirements  for  Decision  Making 

-  Description  of  the  decisions  to  be  made  (relative,  absolute,  probabilistic) 

-  Evaluations  to  be  conducted 

-  Decisions  that  are  probable  based  on  the  evaluation  (opportunities  for  cognitive  errors,  such  as  capture 
error,  will  be  identified  and  carefully  analyzed) 
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•  Requirements  for  Response 

-  Action  to  be  taken 

-  Overlap  of  task  requirements  (serial  vs.  parallel  task  elements) 

-  Frequency 

-  Time  available  for  operator  response  based  on  characteristics  of  plant  response 

-  Temporal  constraints  (task  ordering) 

-  Tolerance  and  accuracy 

-  Operational  limits  of  personnel  performance 

-  Operational  limits  of  machine  and  software 

-  Body  movements  required  by  action  taken 

•  Requirements  for  Feedback 

-  Feedback  required  to  indicate  adequacy  of  actions  taken 

The  task  analysis  should  be  iterative  and  become  progressively  more  detailed  over  the  design  cycle.  Sufficient 
detail  is  necessary  to  identify  information  and  control  requirements  to  enable  specification  of  detailed  requirements 
for  HSI  resources,  such  as  alarms,  displays,  data  processing,  and  controls. 

Based  on  its  use  of  functional  decomposition  and  analysis  of  cognitive  requirements  (information  gathering, 
decision  making,  response,  and  feedback),  NUREG-071 1  is  consistent  with  recent  approaches  to  cognitive  task 
analysis  (e.g.,  Roth  and  Mumaw,  1995). 

Human  Reliability  Analysis 

The  objective  of  the  human  reliability  analysis  (HRA)  of  the  HFE  design  is  to  ensure  that  potential  effects  of 
human  error  on  plant  safety  and  reliability  were  analyzed,  that  human  actions  important  to  plant  risk  were 
identified,  and  that  human  error  mechanisms  were  addressed  in  the  design  of  the  HSIs,  to  minimize  the  likelihood 
of  personnel  error  and  to  provide  for  error  detection  and  recovery  capability. 

The  HRA  element  of  NUREG-071 1  that  is  especially  pertinent  to  developing  information  requirements  is  the  area 
of  risk-significant  human  actions.  The  HRA  will  develop  those  operator  actions  that  are  especially  significant  to 
risk.  These  actions  are  often  associated  with  low-probability  and  high-consequence  sequences  of  events.  These 
actions  may  relate  to  activities  that  only  are  needed  once  certain  failures  and  transients  have  already  occurred.  As 
a  result,  they  would  not  typically  be  a  part  of  task  analyses  that  are  done  during  the  usual  design  process.  Thus, 
consideration  of  these  risk-significant  human  actions  will  often  add  more  controls,  displays,  or  alarms  that  would 
not  have  otherwise  been  included. 
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HSI  Design 

The  HSI  design  process,  as  the  term  is  used  here,  is  the  translation  of  function  and  task  requirements  into  a 
detailed  HSI  design;  that  is,  the  specific  design  of  the  HSI  resources  identified  above  as  well  as  their  integration  in 
CRs  and  other  workplaces.  This  translation  should  be  designed  using  a  structured  methodology  that  guides 
designers  in  the  identification  of  what  information  and  controls  are  required,  the  identification  and  selection  of 
candidate  HSI  approaches,  and  the  detailed  design  of  the  HSIs.  Constraints  imposed  by  the  overall  instrumentation 
and  control  (I&C)  system  should  be  considered  throughout  the  design  process. 

The  HSI  design  methodology  addresses  specific  design  activities,  including: 

•  Development  and  use  of  guidelines  for  HFE  design 

•  Functional  requirement  specification 

•  HSI  concept  design 

•  Detailed  design  and  integration  of  the  HSI 

•  Tests  and  evaluations  of  the  HSI  design  process 

•  Documentation  of  final  design 

At  this  point  the  information  system  requirements  have  been  defined  and  the  representational  aspect  of  the 
information  system  begins  to  be  developed.  Display  formats  are  developed  to  meet  the  functional  requirements 
with  a  design  that  reflects  a  combination  of  past  operational  experience  (what  types  of  formats  were  successful  or 
unsuccessful),  HFE  guidelines,  and  prototype  testing  and  evaluation. 

Section  1,  Information  Display  of  Part  2  of  NUREG-0700,  Rev.  I  provides  guidance  for  information  system  design 
review.  It  deals  primarily  with  the  review  of  visual  displays,  both  text  and  graphics  based.  Guidance  is  provided 
in  top-down  fashion  beginning  with  general  display  guidelines  (Section  I.I),  and  then  finer  levels  of  display 
details.  Section  1 .2,  Display  Formats,  addresses  common  display  forms: 

•  Continuous  Text  Displays 

•  Tables  and  Lists 

•  Data  Forms  and  Fields 

•  Bar  Charts  and  Histograms 

•  Graphs 

•  Pie  Charts 

•  Flowcharts 

•  Mimics  and  Diagrams 
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•  Maps 

•  Graphic  Instrument  Panels 

•  Speech  Displays 

In  addition  to  the  display  guidance  in  NUREG-0700,  guidance  for  group-view  displays,  such  as  is  in  advanced  CR 
wall  panel  displays,  is  available  (Stubler  and  O’Hara,  1996). 

A  display  format  can  be  decomposed  into  a  set  of  basic  elements,  such  as  alphanumeric  characters,  icons,  symbols, 
color,  highlighting,  abbreviations,  labels,  coding,  and  presentation  of  numeric  data.  These  are  addressed  in 
Section  1 .3,  Display  Elements.  Section  1 .4  contains  guidelines  for  reviewing  the  quality  of  data  and  display  update 
rate.  Section  1 .5,  Display  Devices,  addresses  the  hardware  aspects  of  displays.  Review  guidelines  are  given  for 
display  devices,  such  as  video  display  units,  large  panel  displays,  and  hard-copy  display  devices  (printers  and 
plotters). 

There  are  two  significant  shortcomings  in  the  treatment  of  information  representation:  display  formats  do  not 
correspond  to  these  generic  formats,  and  the  organization  of  displays  in  a  display  network.  For  novel  formats,  i.e., 
formats  not  addressed  in  NUREG-0700,  it  was  recommended  that  they  be  evaluated  using  the  general  display 
guidelines  and  the  guidelines  in  Section  1.3,  Display  Elements.  While  this  provides  some  support  for  design 
review,  a  review  limited  to  these  guidelines  would  fail  to  address  important  aspects  of  many  novel  designs,  such  as 
the  relationship  between  their  dynamic  graphic  elements  and  the  underlying  process  dynamics  that  they  are 
intended  to  represent.  The  organization  of  displays  in  an  information  network  was  not  addressed  at  all. 

HSI  designs  should  be  tested  and  evaluated  throughout  their  development.  This  is  especially  important  for  designs 
that  are  not  well  defined  by  HFE  guidance.  NUREG-071 1  addresses  the  methodology'  used  for  testing. 

Verification  and  Validation  (V&V) 

V&V  of  HSI  designs  should  be  conducted  to  ensure  the  acceptability  of  the  final  design  and  to  address  any 
remaining  issues.  Because  no  one  method  is  likely  to  be  sufficiently  comprehensive,  a  series  of  analyses  may  be 
done. 

HSI  Task  Support  Verification.  This  evaluation  verifies  that  the  HSI  supports  all  identified  personnel  task 
requirements.  This  is  determined  by  comparing  the  HSI  design  elements  to  the  criteria  defined  by  function  and 
task  analysis  requirements.  Problems  are  identified  for  (1)  personnel  task  requirements  that  are  not  fully  supported 
by  the  HSI,  and  (2)  the  presence  of  HSI  components,  which  may  not  be  needed  to  support  personnel  tasks. 

HFE  Design  Verification.  This  evaluation  verifies  that  the  HSI  is  designed  and  implemented  to  account  for 
capabilities  and  limitations  of  humans.  This  is  determined  by  comparing  the  detailed  HSI  design  to  the  criteria 
defined  by  HFE  guidelines.  Problems  are  identified  if  the  design  is  inconsistent  with  HFE  guidelines. 

Integrated  System  Validation.  This  evaluation  validates  that  the  integrated  HSI  design  enables  tasks  required  for 
safe  operation  to  be  performed  without  imposing  excessive  workload.  The  performance  criteria  are  derived  from 
task,  risk,  and  engineering  analyses.  Problems  are  identified  if  performance  criteria  are  not  achieved,  or  if  the  HSI 
imposes  a  high  workload  on  plant  personnel. 
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5.2  Cognitive  Tasks  Supported  by  Information  Systems 

The  plant  information  systems  provide  the  basis  for  operators  to  perform  their  role  in  the  plant.  In  this  section,  we 
consider  the  general  tasks  that  the  information  system  must  support.  This  is  important  in  scoping  out  the  range  of 
cognitive  activities  that  must  be  addressed  in  the  design  of  information  systems.  This  is  not  only  important  when 
considering  requirements  for  design,  but  also  for  evaluating  the  results  of  research  on  displays  for  generalizability 
to  the  full  range  of  NPP  operations. 

The  operator’s  role  in  a  NPP  is  that  of  a  supervisory  controller,  i.e.,  the  plant’s  performance  is  the  result  of  the 
interaction  of  human  and  automatic  control.  Reason  (1990)  called  this  a  complex  multiple-dynamic  configuration; 
in  such  settings,  situation  assessment  and  response  planning  can  be  difficult  when  things  go  wrong.  In  addition  to 
failures  of  plant  process,  the  automatic  control  systems  and  HSI  also  can  fail.  Thus,  personnel  must  respond  to 
failures  in  the  plant  and  to  the  interfaces  that  communicate  failures.  One  of  the  most  significant  aspects  of  the  HSI 
in  responding  to  process  failures  is  the  plant  information  system. 

The  operator’s  impact  on  plant  functions,  processes,  systems,  and  components  is  mediated  by  a  causal  chain  from 
the  operator’s  physiological  and  cognitive  processes,  to  operator  task  performance,  and  ultimately  to  plant 
performance  through  the  operator’s  manipulation  of  the  plant’s  HSI.  Design  of  HSI,  including  displays,  impacts 
plant’s  performance  through  personnel  tasks  in  support  of  plant  operations.  The  operator’s  role  involves  two  types 
of  tasks,  primary  tasks  and  secondary  tasks.  Primary  tasks  are  those  the  operator  performs  as  part  of  the  functional 
role  of  supervising  the  plant.  Operators  may  be  required  to  take  an  action  in  support  of  the  plant’s  performance  of 
a  higher-level  function.  Even  when  operators  are  not  required  to  take  an  explicit  action,  they  must  monitor  the 
performance  of  automatic  systems  and  intervene  when  they  fail  or  function  unacceptably. 

Primary  tasks  involve  several  generic  cognitive  tasks,  i.e.,  situation  assessment,  monitoring  and  detection,  response 
planning,  and  response  implementation  (Figure  5.1).  When  addressing  primary  tasks,  these  generic  cognitive 
tasks  are  discussed  rather  than  the  detailed  specific  tasks,  such  as  monitoring  steam  flow,  starting  pumps,  and 
aligning  valves. 

Secondary  tasks  are  those  the  operator  must  perform  when  interfacing  with  the  HSIs  or  job  performance  aids,  but 
which  are  not  directed  to  the  primary  task  (O’Hara,  Stubler,  and  Nasta,  1997).  Secondary  tasks  include  navigating 
through  an  information  system  and  manipulating  windows  on  a  VDU.  To  adequately  perform  their  tasks,  both 
primary  and  secondary,  operators  use  their  information  processing  resources,  such  as  attention,  reasoning,  and 
memory. 

The  design  of  the  information  system  affects  each  of  the  operator’s  generic  cognitive  primary  tasks  and  the 
secondary  tasks.  In  this  section,  we  address  situation  assessment,  monitoring,  response  planning,  and  response 
implementation. 

5.2.1  Situation  Assessment 

When  faced  with  an  abnormal  occurrence,  operators  actively  try  to  construct  a  coherent,  logical  explanation  to 
account  for  their  observations.  This  cognitive  activity,  situation  assessment,  involves  two  related  concepts:  the 
situation  model  and  the  mental  model.  Operators  develop  and  update  a  mental  representation  of  the  factors  known, 
or  hypothesized,  to  be  affecting  the  plant’s  state  at  a  given  point  in  time.  The  mental  representation  resulting  from 
situation  assessment  may  be  referred  to  as  a  situation  model ,  the  person’s  understanding  of  the  specific  current 
situation.  The  situation  model  is  constantly  updated  as  new  information  is  received. 
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Figure  5.1  Generic  Primary  Tasks  of  a  Supervisory  Controller 


To  construct  a  situation  model,  operators  use  their  general  knowledge  and  understanding  about  the  plant  and  how 
it  operates  to  interpret  the  information  they  observe  and  understand  its  implications.  Limitations  in  knowledge 
may  result  in  incomplete  or  inaccurate  situation  models.  The  general  knowledge  governing  the  performance  of 
highly  experienced  individuals  may  be  referred  to  as  a  mental  model,  which  constitutes  the  operator’s  internal 
representation  of  the  physical  and  functional  characteristics  of  the  system  and  its  operation.  Mental  models  may 
not  always  be  fully  accurate  or  complete  (Woods  et  al.,  1994). 

The  mental  model  is  built  up  through  formal  education,  system-specific  training,  and  operational  experience.  It  is 
represented  in  the  knowledge  bases  of  long-term  memory.  An  accurate  mental  model  is  generally  considered  a 
defining  characteristic  of  expert  performance  (e.g.,  Wickens,  1984;  Bainbridge,  1986;  Moray  et  al.,  1986; 
Rasmussen,  1983;  Sheridan,  1976)  and  is  extremely  important  to  many  aspects  of  information  processing 
(discussed  below).  The  mental  model  is  thought  to  drive  skill-based  processing,  control  rule-based  activity  through 
the  mediation  of  the  operator’s  conscious  effort  in  working  memory,  and  provide  the  substantive  capability  to 
reason  about  and  predict  future  plant  states,  which  is  required  of  knowledge-based  processing  (Rasmussen,  1983). 

The  distinctions  between  the  mental  and  situation  models  reflect  their  cognitive  underpinnings  in  long-term  and 
working  memory.  The  mental  model  is  relatively  permanent.  By  contrast,  an  operator’s  situation  model  is  the 
current  interpretation  of  the  plant’s  status,  and  therefore,  can  be  changed. 


NUREG/CR-6633 


5-8 


5  DEVELOPMENT  OF  TECHNICAL  BASIS 


When  the  operator’s  situation  model  is  an  accurate  reflection  of  the  plant’s  actual  state,  an  operator  is  said  to  have 
good  situation  awareness.  Thus,  the  accuracy  of  situation  awareness  is  a  function  of  the  degree  of  correlation 
between  the  operator’s  situation  model  and  the  actual  plant  conditions  at  any  given  time.  An  operator  can  have  a 
good  mental  model  (e.g.,  knowledge  of  how  the  plant  functions),  but  poor  situation  awareness  because  the  situation 
model  does  not  match  the  current  plant  conditions.  The  process  of  situation  assessment  has  been  identified  as  the 
single  most  important  factor  in  improving  effectiveness  of  the  crew  in  complex  systems  (Endsley,  1988). 

For  an  experienced,  well-trained  operator,  when  the  HSI  can  provide  information  that  readily  maps  to  knowledge 
in  the  operator’s  mental  model,  an  accurate  situation  model  is  easily  developed.  To  the  extent  that  an  easy  match 
cannot  be  made  between  plant  information  and  the  situations  defined  in  the  mental  model,  situation  assessment 
requires  more  working  memory  and  attention,  and  cognitive  workload  will  be  high  (Endsley,  1993,  1995;  Fraker, 
1988).  However,  in  addition  to  supporting  situation  assessment,  working  memory  also  must  support  other 
activities,  such  as  the  selection  and  implementation  of  operator  actions.  Accordingly,  if  other  tasks  place  high 
demands  on  working  memory,  situation  awareness  may  suffer.  On  the  other  hand,  the  demands  placed  on  working 
memory  in  maintaining  situation  awareness  can  be  lessened  if  the  situation  is  familiar  (based  on  training  or 
experience),  and  therefore,  can  be  identified  with  a  representation  stored  in  long-term  memory.  In  this  case,  it 
would  not  be  necessary  for  the  operator  to  maintain  in  working  memory  each  detail  of  the  situation. 

Situation  awareness  and  cognitive  workload  may  vary  inversely  under  complex,  somewhat  ambiguous  situations. 

For  example,  under  unfamiliar  or  otherwise  difficult  conditions,  high  cognitive  workload  may  be  associated  with 
decreased  situation  awareness  This  may  be  due  to  a  lack  of  available  attentional  resources  for  analyzing  the 
situation.  However,  as  Endsley  (1993)  points  out,  situation  awareness  and  cognitive  workload,  while  interrelated, 
may  vary  independently.  For  example,  a  task  may  be  intensive,  but  readily  recognizable.  Situation  awareness 
requires  the  expenditure  of  cognitive  resources  that  contribute  to  workload,  but  it  is  not  the  only  cognitive  activity 
requiring  such  resources. 

Thus,  mental  models  enable  operators  to  engage  in  situation  assessment  and  to  establish  situation  models.  Another 
important  aspect  of  mental  models  is  that  they  enable  operators  to  make  predictions  and  form  expectations  that,  in 
turn,  guide  monitoring  and  affect  the  interpretation  of  information.  This  is  a  general  characteristic  of  human 
information  processing;  it  is  a  synthesis  of  “bottom-up”  processing  (what  an  operator  perceives  from  the 
environment)  and  “top-down”  processing  (what  an  operator  expects)  (Neisser,  1967).  An  example  of  bottom-up 
processing  occurs  during  a  disturbance  when  an  operator  monitors  the  HSI  and  processes  data  from  the  interface  to 
determine  what  is  wrong.  At  the  same  time,  these  data  are  used  to  formulate  hypotheses  or  expectations  about  the 
status  of  the  plant,  which  serve  to  structure  the  perceptual  process  and  data  gathering  occurring  at  lower  levels. 

This  is  top-down  processing.  Both  contribute  to  the  operator’s  interpretation  of  the  situation. 

The  ability  to  make  predictions  using  a  mental  model  that  is  based  on  the  current  situation  model  enables  the 
operator’s  performance  to  become  more  “open-loop”  (Moray,  1986).  “Open-loop”  in  this  context  means  that 
behavior  becomes  less  driven  by  feedback  and  more  governed  by  the  operator’s  prediction  of  future  system  behavior 
and  the  desired  goal  state.  A  NPP  mental  model  includes  such  knowledge  as  the  physical  interconnections  among 
plant  systems  to  predict  flow  paths  (e.g.,  considering  piping  and  valve  interconnections  to  figure  out  how  water 
from  one  system  could  get  into  another)  and  knowledge  of  mass  and  energy  changes  in  one  system  to  predict  the 
effect  on  a  second  system  (e.g.,  predicting  the  effect  that  changes  in  secondary  side  steam  generator  levels  and 
temperatures  will  have  on  primary  system  cooldown).  While  mental  models  provide  the  principles  upon  which 
predictions  can  be  made,  the  situation  model  provides  the  starting  point  and  becomes  the  basis  from  which 
expectations  are  developed  about  events  that  should  be  happening  at  the  same  time,  how  events  should  evolve  over 
time,  and  effects  that  may  occur  in  the  future. 
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The  operator’s  expectations  of  the  near-term  future  state  of  the  plant  are  used  to  guide  the  sampling  of  indicators  to 
confirm  the  inference  (Bainbridge,  1 974).  Expectations  are  used  to  search  for  evidence  to  confirm  the  current 
situation  model  and  to  explain  observed  symptoms.  If  a  new  symptom  is  observed  that  is  consistent  with  operator 
expectations,  a  ready  explanation  for  the  finding  will  be  developed,  yielding  greater  confidence  in  the  situation 
model. 

While  the  mental  model  allows  prediction  and  expectancy  to  guide  control  responses,  expectancy  also  can  make 
detecting  subtle  system  failures  difficult  ( Wickens  and  Kessel,  1981).  When  a  new  symptom  is  inconsistent  with 
an  operator’s  expectation,  the  operator  may  discount  or  misinterpret  it  to  make  it  consistent  with  the  expectations 
derived  from  the  current  situation  model.  For  example,  an  operator  may  fail  to  detect  key  signals,  or  detect  them 
but  misinterpret  or  discount  them,  because  of  an  inappropriate  understanding  of  the  situation  and  the  expectations 
derived  from  that  understanding.  That  is,  operators  tend  to  ignore  or  discount  symptoms  that  are  not  consistent 
with  their  situation  model.  However,  if  the  new  symptom  is  recognized  as  an  unexpected  plant  behavior,  the  need 
to  revise  the  situation  model  will  become  apparent.  In  that  case,  the  symptom  may  trigger  situation  assessment 
activity  to  search  for  a  better  explanation  of  the  current  observations.  In  turn,  situation  assessment  may  involve 
developing  a  hypothesis  for  what  might  be  occurring,  and  then  searching  for  confirmatory  evidence  in  the 
environment.  Thus,  situation  assessment  activities  can  result  in  detecting  abnormal  plant  behavior  that  might  not 
otherwise  have  been  observed,  detecting  plant  symptoms  and  alarms  that  may  have  otherwise  been  missed,  and 
identifying  problems,  such  as  sensor  failures  or  plant  malfunctions. 

The  situation  model  is  constantly  updated  as  new  information  is  received  and  a  person’s  understanding  of  a 
situation  changes.  In  NPP  applications,  maintaining  and  updating  a  situation  model  entails  keeping  track  of  the 
changing  factors  that  influence  plant  processes,  including  faults,  operator  actions,  and  automatic  system  responses. 

The  importance  of  mental  and  situation  models,  and  the  expectations  that  are  based  on  them,  cannot  be 
overemphasized.  They  not  only  govern  situation  assessment,  but  also  play  an  important  role  in  guiding 
monitoring,  using  procedures  and  formulating  response  plans,  and  implementing  responses. 

On  the  basis  of  the  above  discussion,  several  factors  can  be  identified  that  make  the  process  of  situation  assessment 
difficult:  unanticipated  events,  a  poor  mental  model,  a  poorly  designed  information  system,  and  divided  attention. 

•  Unanticipated  events  -  Mental  models  are  established  through  training  and  experience.  Situation  assessment 
involves  matching  current  information  to  those  models.  Therefore,  situations  or  events  that  were  not  analyzed 
by  designers,  for  which  operators  were  not  trained  and  have  not  experienced,  will  be  difficult  to  assess. 

Vicente  and  Rasmussen  (1992)  suggested  that  a  significant  design  challenge  is  to  develop  a  HSI  to  support 
operators  in  handling  unanticipated  events.  Since  responses  to  NPP  abnormal  and  emergency  events  are 
analyzed  carefully  in  advance  to  design  systems  to  handle  the  disturbance  (e.g.,  engineered  safety  features)  and 
operator  responses  (e.g.,  emergency  operating  procedures  [EOPs]),  the  unplanned  and  unanticipated  events 
pose  the  greatest  challenge  to  the  human-machine  system.  As  Vicente  and  Rasmussen  (1992)  noted,  “The 
inescapable  conclusion  is  that  unanticipated  events  can  and  do  happen  in  large-scale  industrial  systems.  In 
fact,  they  are  the  major  cause  of  life-threatening  accidents”  (p.  589). 

•  Poor  mental  model  -  Even  when  events  are  anticipated,  failures  in  training  or  inadequate  experience  can 
result  in  mental  models  that  are  not  adequate  to  support  situation  assessment. 

•  Poor  information  system  design  -  Even  when  events  occur  that  are  familiar  and  well  understood  by  operators, 
they  must  recognize  them.  As  noted  above,  this  is  accomplished  by  a  match  between  information  provided  by 
the  HSI  and  the  operator’s  mental  model.  Information  systems  may  not  support  such  a  mapping  very 
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effectively.  For  example,  relevant  information  may  be  lost  in  an  information-dense  display  or  spread  out 
across  many  displays.  Even  when  all  the  relevant  information  is  available,  it  may  be  presented  in  a  way,  such 
as  individual  parameters,  that  operators  must  mentally  process  to  get  it  to  a  level  where  it  is  able  to  be  mapped 
to  the  mental  model. 

•  Divided  attention  -  Maintenance  of  situation  awareness  is  disrupted  by  shifting  attention  between  different 
tasks  (Gaba  and  Howard,  1995).  Attention  is  directed  away  from  situation  assessment  to  engage  in  other 
primary  tasks,  such  as  communications  with  coworkers,  or  secondary  tasks,  such  as  retrieving  a  display. 

5.2.2  Monitoring  and  Detection 

Monitoring  and  detection  refer  to  the  activities  involved  in  extracting  information  from  the  environment. 
Monitoring  is  checking  the  state  of  the  plant  to  determine  whether  the  systems  are  operating  correctly,  including 
checking  parameters  indicated  on  the  CR  panels,  monitoring  parameters  displayed  by  the  process  computer, 
obtaining  verbal  reports  from  operators  in  the  plant  areas,  and  sending  operators  to  areas  of  the  plant  to  check  on 
equipment  Detection  is  the  operator’s  recognition  that  something  is  not  operating  correctly  and  that  an 
abnormality  exists. 

In  a  highly  automated  plant,  much  of  what  supervisory  controllers  do  involves  monitoring.  For  example,  operators 
must  monitor 

•  Normal  conditions  to  determine  that  what  is  expected  to  happen  does 

•  Results  of  actions  (feedback) 

•  Normal  and  safety  indications  for  changes  or  disturbances 

•  Performance  of  automated  systems 

•  Problematic  equipment 

•  Activities  of  coworkers  (test  and  maintenance) 

Ironically,  despite  the  fact  that  monitoring  is  such  a  central  cognitive  activity,  the  monitoring  by  supervisory 
controllers  of  computer-based  systems  is  often  “unstructured”  (Thurman,  1997).  Operators  are  advised  to  monitor 
everything  and  given  little  guidance  on  how  to  accomplish  this.  Thus,  monitoring  lacks  goal  direction  and 
important  information  can  be  missed.  Further,  display  designs  often  fail  to  support  monitoring  because  designers 
simply  take  all  available  status  information  and  develop  a  set  of  displays  structured  around  a  physical 
representation  of  the  system  (Thurman,  1997).  The  cognitive  basis  for  monitoring  is  discussed  below. 

Monitoring  and  detection  are  influenced  by  two  factors:  the  characteristics  of  the  environment  and  the  operator’s 
knowledge  and  expectations.  These  factors  lead  to  two  types  of  monitoring:  data-driven  and  model-driven. 
Monitoring  that  is  driven  by  characteristics  of  the  environment  is  often  referred  to  as  data-driven  monitoring. 
Data-driven  monitoring  is  affected  by  the  salience  of  the  information’s  presentation  (e.g.,  size,  color,  and 
loudness).  For  example,  alarm  systems  are  basically  automated  monitors  that  are  designed  to  influence  data-driven 
monitoring  by  using  aspects  of  physical  salience  to  direct  attention.  Auditory  alerts,  flashing,  and  color  coding  are 
examples  of  physical  characteristics  that  enable  operators  to  quickly  identify  an  important  new  alarm.  Data-driven 
monitoring  also  is  influenced  by  the  behavior  of  the  information  being  monitored,  such  as  the  bandwidth  and  rate 
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of  change  of  the  information  signal.  For  example,  observers  more  frequently  monitor  a  signal  that  is  rapidly 
changing. 

Model-driven  monitoring  is  initiated  by  operators  based  on  their  knowledge  and  expectations  about  the  most 
important  sources  of  information.  This  type  of  monitoring  also  is  referred  to  as  knowledge-driven  monitoring. 
Knowledge-driven  monitoring  can  be  viewed  as  active  monitoring  in  that  the  operator  is  not  merely  responding  to 
characteristics  of  the  environment  that  “shout  out”  as  an  alarm  system  does,  but  is  deliberately  directing  attention 
to  areas  of  the  environment  that  are  expected  to  provide  specific  information. 

Model-driven  monitoring  may  be  initiated  by  several  factors.  First,  it  may  be  guided  by  operating  procedures  or 
standard  practice  (e.g.,  control  panel  walk-downs  that  accompany  shift  turnovers).  Second,  it  can  be  triggered  by 
situation  assessment  or  response  planning  activities,  and  therefore  is  strongly  influenced  by  a  person’s  current 
situation  model.  The  situation  model  allows  the  operator  to  direct  attention  and  focus  monitoring  effectively. 
However,  such  a  monitoring  strategy  also  can  lead  operators  to  miss  important  information.  For  example,  an 
incorrect  situation  model  may  lead  an  operator  to  focus  attention  in  the  wrong  place,  to  fail  to  observe  a  critical 
finding,  or  to  misinterpret  or  discount  an  indication. 

An  operator  is  faced  with  an  environment  containing  more  variables  than  can  be  realistically  monitored. 

Monitoring  is  challenging  because  there  are  a  large  number  of  potentially  relevant  things  to  attend  to  at  any  point 
in  time  and  that  the  operator  must  determine  what  information  is  worth  pursuing  within  a  constantly  changing 
environment  (Vicente  et  al.,  1997).  In  this  situation,  monitoring  requires  the  operator  to  decide  what  to  monitor 
and  when  to  shift  attention  elsewhere.  These  decisions  are  strongly  influenced  by  an  operator’s  current  situation 
model,  which  guides  the  allocation  of  attentional  resources  to  sampling  data  from  the  environment  based  on  its 
statistical  properties;  i.e.,  expected  probability  and  correlation.  The  operator’s  ability  to  develop  and  effectively  use 
knowledge  to  guide  monitoring  relies  on  an  understanding  of  the  current  state  of  the  process.  As  cognitive 
workload  increases,  monitoring  strategies  become  less  thorough  and  the  capability  to  detect  particular  failures 
decreases  (Ephrath  and  Young,  1981). 

As  discussed  above,  under  normal  conditions,  situation  assessment  is  accomplished  by  mapping  the  information 
obtained  in  monitoring  to  elements  in  the  situation  model.  For  experienced  operators,  this  comparison  is  relatively 
effortless  and  requires  little  attention.  During  unfamiliar  conditions,  however,  the  process  is  considerably  more 
complex.  The  first  step  in  realizing  that  the  current  plant  conditions  are  not  consistent  with  the  situation  model  is 
to  detect  a  discrepancy  between  information  representing  the  current  situation  and  information  derived  from 
monitoring.  This  process  is  facilitated  by  the  alarm  system,  which  helps  to  direct  the  attention  of  a  plant  operator 
to  an  off-normal  situation. 

When  determining  whether  or  not  a  signal  is  significant  and  worth  further  investigation,  operators  examine  the 
signal  in  the  context  of  their  current  situation  model.  They  must  judge  whether  the  anomaly  indicates  a  real 
abnormality  or  an  instrumentation  failure.  They  will  then  assess  the  likely  cause  of  the  abnormality  and  evaluate 
the  importance  of  the  signal  in  determining  their  next  action. 

Monitoring  and  detection  were  described  in  terms  of  signal  detection  theory  (SDT)  (Green  and  Swets,  1988). 
Process  control  operators  are  in  a  monitoring  environment  that  was  described  in  SDT  terms  as  an  alerted-monitor 
system  (Sorkin  et  al.,  1985;  Sorkin  and  Woods,  1988).  Such  a  system  is  composed  of  an  automated  monitor  and  a 
human  monitor,  an  operator.  The  automated  monitor  is  the  alarm  system  which  monitors  the  system  to  detect 
off-normal  conditions.  When  a  plant  parameter  exceeds  the  criterion  of  the  automated  monitor,  the  operator  is 
alerted  and  must  then  detect,  analyze,  and  interpret  the  signal  as  a  false  alarm,  or  a  true  indication  of  a  plant  upset. 
The  operator  also  can  assess  plant  parameters  independently  of  the  automated  monitor  (the  alarm  system).  Both 
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the  operator  and  alarm  system  have  their  own  specific,  signal  detection  parameter  values  for  sensitivity  (d1)  and 
response  criterion.  The  response  criterion  refers  to  the  amount  of  evidence  that  is  needed  before  an  operator  will 
conclude  that  a  signaled  event  is  actually  present;  this  is  sometimes  referred  to  as  response  bias  since  it  describes 
an  operator’s  degree  of  conservatism.  Sensitivity  refers  to  the  resolution  of  the  system  and  determines  the  ease 
with  which  signals  (represented  as  a  statistical  distribution)  can  be  distinguished  from  signals  and  noise  (also 
represented  as  a  distribution). 

SDT  research  has  many  implications  for  understanding  how  operators  process  information  during  a  disturbance. 
First,  the  response  criterion  is  affected  by  expectancy,  i.e.,  the  expected  probability  that  an  event  will  occur  and  the 
payoff  structure  (rewards  and  penalties  for  making  correct  and  incorrect  detections,  respectively).  While  alarms 
can  occur  frequently,  significant  off-normal  events  in  NPPs  typically  have  a  low  probability  of  occurring. 
Therefore,  operators  have  low  expectancy  about  their  actual  occurrence,  which  creates  a  conflict  between  the  cost 
to  productivity  for  falsely  taking  an  action  that  shuts  down  the  reactor  versus  the  cost  for  failing  to  take  a  warranted 
action.  In  a  real-world  system,  disturbances  have  a  low  probability,  and  operators  therefore  rely  on  redundant  and 
supplemental  information  to  confirm  the  alarmed  condition.  Upon  verification  of  several  confirmatory  indicators, 
the  operator  can  accept  the  alarm  information  as  indicating  an  actual  off-normal  condition  (compared  with  a 
spurious  condition). 

There  are  two  types  of  anomalies:  (I)  deviations  from  desired  system  function,  referred  to  as  abnormal  findings, 
and  (2)  deviations  from  the  operator’s  situation  model,  referred  to  as  unexpected  findings.  The  different  kinds  of 
anomalies  lead  to  different  follow-up  reasoning  and  monitoring  behavior: 

•  Abnormal  findings  lead  to  information  processing  about  how  to  cope  with  the  disturbance  (response  planning) 
and  to  monitoring  behavior  to  see  if  responses  to  coping  occurred  as  expected,  and  whether  they  are  having  the 
desired  effect. 

•  Unexpected  findings  or  process  behavior  lead  to  situation  assessment  activity  and  knowledge-driven 
monitoring  to  explain  the  finding. 

Failures  in  monitoring  can  include  failing  to  observe  parameters,  misunderstanding  their  significance,  or  failing  to 
obtain  necessary  reports  from  plant  areas.  Failures  in  detection  can  include  failing  to  recognize  an  abnormality 
despite  appropriate  monitoring.  An  error  in  monitoring  or  detection  can  lead  to  the  operator’s  failure  to  respond  to 
the  event,  or  at  least,  failure  to  respond  within  the  required  time. 

Two  aspects  of  plant  design  are  significant  to  monitoring:  HS1  technology  and  plant  sophistication.  HSI 
technology  has  a  significant  impact  on  monitoring.  In  conventional  CRs,  displays  and  controls  are  predominantly 
spatially-dedicated.  They  have  fixed  locations  that  cannot  be  changed  in  form  or  function.  In  such  situations, 
operators  develop  scanning  patterns  to  support  monitoring.  However,  even  with  spatial  dedication,  monitoring  and 
search  can  be  difficult  because  information  can  be  physically  hidden  from  view  (Barber,  1996),  e.g.,  when  displays 
are  obscured  by  tags  (as  happened  during  the  TMI  event)  or  located  away  from  the  main  CR.  One  of  the  major 
values  of  the  detailed  design  reviews  of  CRs  in  the  1980s  was  making  the  search  process  easier  (Van  Cott,  1997). 
Improvements  that  facilitated  interface  management  and  reduced  the  chance  for  error  were  evident  in  the 
modifications  based  on  NUREG-0700.  These  included  better  organization  of  indicators  and  controls,  improved 
labeling,  and  improved  demarcation  of  the  relationships  between  indicators  and  controls  through  the  use  of  board 
mimics.  However,  there  was  little  actual  manipulation  of  the  interface  -  what  Vicente,  Mumaw,  and  Roth  (1997) 
referred  to  as  “HSI  degrees  of  freedom.”  The  CR  information  and  controls  were  fixed  and  spatially  dedicated  in 
the  way  the  HSI  designer  felt  was  most  appropriate. 
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By  contrast  to  conventional  HSIs,  advanced  HSIs  are  flexible  because  they  can  be  configured  and  function  in 
various  operating  modes.  These  interfaces  have  several  degrees  of  freedom.  For  example,  based  on  observation  of 
operators  monitoring  a  relatively  advanced  NPP,  Vicente  et  al.  (1997)  found  that  operators  could  decide  “...where 
to  put  a  given  display,  what  the  time  scale  on  a  trend  graph  should  be,  what  the  range  on  a  trend  graph  should  be, 
what  form  to  present  a  variable  in  (e.g.,  trend,  bar,  bar-trend,  digital),  what  variables  should  be  graphed  together, 
and  so  on.  Not  only  can  the  operator  make  these  decisions,  but  they  can  make  them  over  and  over  again  in  the 
sense  that  these  presentation  parameters  can  be  readily  changed  in  a  moment.”  However,  operators  tended  to 
ignore  the  flexibility  offered  by  the  system.  Instead,  they  selected  a  set  of  displays  that  they  felt  gave  them  a  good 
overview  of  the  plant,  and  left  those  on  the  VDUs,  which  was  done,  to  some  extent,  at  the  expense  being  able  to 
access  more  detailed  views  of  the  plant  that  may  have  been  available.  This  was  attributed,  in  part,  to  the  effort 
involved  in  paging  through  displays,  a  task  that  operators  felt  interfered  with  their  ability  to  understand  the  overall 
status  of  the  plant. 

With  regard  to  monitoring,  Vicente  et  al.  (1997)  commented: 

Computer-based  HSIs  can  make  it  difficult  to  find  information  and  act  on  the  system  in  comparison  with  conventional 
plants.. .In  a  computer-based  control  room,  there  is  a  significant  increase  in  the  amount  of  knowledge  required  to 
operate  the  interface.  Part  of  these  demands  arise  from  the  fact  that  information  is  presented  serially.  Thus, 
operators  need  to  know  how  to  bring  up  the  information  they  want  to  display  on  CRTs.  The  other  part  of  the  increase 
in  knowledge  demands  arises  from  the  fact  that  the  computer-based  displays  are  much  more  flexible  than  the  hard¬ 
wired  instruments.  The  same  information  can  be  displayed  at  different  time  scales,  at  different  ranges,  in  different 
locations,  in  different  forms,  and  in  different  groupings.  Consequently,  the  operator  has  to  have  much  more 
knowledge  about  the  interface  (not  the  unit)  to  resolve  the  degrees  of  freedom  offered  by  the  flexible  design  of  the 
system..  On  the  one  hand,  one  might  argue  that  the  flexibility  provided  by  the  computer-based  medium  should  result 
in  a  performance  improvement  because  it  allows  operators  to  view  information  in  a  form  that  is  tailored  to  different 
types  of  contexts.  This  should  reduce  the  need  for  the  work-arounds  that  operators  engage  in  traditional  control 
rooms  to  facilitate  monitoring.  On  the  other  hand,  one  could  just  as  well  argue  that  this  flexibility  comes  at  the  price 
of  an  increase  in  the  time  and  effort  that  operators  spend  manipulating  the  interface  rather  than  monitoring  the  unit. 

The  problems  associated  with  information  retrieval  and  control  manipulation  will  be  compounded  by  the  fact  that  a 
small  set  of  control  actions,  such  as  positioning  a  cursor,  will  be  required  for  all  CR  activities.  Vicente  et  al. 

(1997)  make  a  similar  point  for  monitoring.  Whereas  in  a  conventional  CR,  monitoring  was  done  primarily  with 
the  eyes,  in  a  computer-based  CR,  it  is  done  with  the  hands,  which  means  that  considerable  manipulation  of  the 
computer-input  device  is  needed  to  page  through  displays. 

Monitoring  is  also  affected  by  the  technological  sophistication  of  the  plant  design.  In  fact,  the  classical  view  of 
monitoring  is  that  it  is  a  boring  and  tedious  task  that  leads  to  vigilance  difficulties  associated  with  lack  of  cognitive 
stimulation.  While  this  may  be  true  in  some  systems,  recent  research  in  nuclear  plants  has  shown  that  routine 
monitoring  can  be  difficult  (Mumaw  et  al.,  1996;  Vicente  et  al.,  1997).  Some  factors  contributing  to  the  difficulty 
were  identified  by  Mumaw  et  al.  (1996)  following  a  study  of  monitoring  performance  in  two  NPPs.  The  factors 
included  the  following: 

•  System  complexity  -  There  are  thousands  of  components  which  interact  in  different  situations,  thus  it  can  be 
difficult  to  understand  the  implications  of  each. 

•  System  reliability  -  Even  though  the  number  of  components  is  large  and  they  are  highly  reliable,  at  any  one 
moment  there  are  many  that  are  out  of  service  or  not  working  properly.  Operators  must  factor  these 
considerations  into  their  situation  assessment. 
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•  Display  design  -  Given  the  large  number  of  parameter  displays,  it  is  often  difficult  to  detect  when  an  abnormal 
situation  has  occurred.  There  are  several  reasons  for  this.  First,  interpretation  of  displayed  parameter  values 

is  memory  intensive.  Displays  do  not  give  clear  referent  values  and  few  aids  are  provided  to  support  recall  of 
recent  value.  In  addition,  few  emergent  features  are  presented  to  enable  operators  to  rapidly  identify  higher- 
level  information  from  the  displays  provided.  Complicating  the  use  of  the  displays  are  the  same  reliability 
issues  as  for  other  components,  i.e.,  at  any  time,  there  may  be  displays  that  do  not  work  properly.  Failed 
displays  also  can  be  difficult  to  detect. 

•  Automation  design  -  The  operation  of  and  feedback  from  automated  systems  is  not  well  represented  in  the 
display  system.  For  example,  the  actual  status  of  components  being  automatically  controlled  is  not  displayed. 
(Murphy  and  Mitchel  [1986]  identified  a  number  of  effects  on  operator’s  cognition  of  automation  and  their 
implications  for  display  design). 

These  factors  introduce  uncertainties  that  make  monitoring  difficult.  In  fact,  Mumaw  et  al.  concluded  that 
monitoring  during  normal  NPP  operations  are  “better  cast  as  a  problem-solving  activity  than  a  vigilance  task” 

(1996,  p.  30).  It  is  a  complex  situation  assessment  activity  dependent  on  imperfect,  sometimes  nonfunctional, 
displays  depicting  thousands  of  parameters. 

Vicente,  Mumaw,  and  Roth  (1997)  examined  the  generalizability  of  these  factors  to  a  more  advanced  CR.  While 
they  found  the  overall  reliability  and  trustworthiness  of  the  instrumentation  to  be  improved,  the  same  basic 
problems  existed.  In  addition,  they  noted  a  new  problem  associated  with  a  CRT-based  information  system,  the 
keyhole  effect,  which  adds  difficulty  in  monitoring. 

5.2.3  Response  Planning 

Response  planning  refers  to  deciding  upon  a  course  of  action  to  address  an  event.  Response  planning  can  be  as 
simple  as  selecting  an  alarm  response  procedure  or  Emergency  Operating  Procedure  (EOP),  or  it  may  involve  more 
thoroughly  developing  a  plan  in  circumstances  where  existing  procedures  have  proved  incomplete  or  ineffective. 

In  general,  response  planning  involves  the  operators  using  their  situation  model  of  the  current  plant  state  to 
identify  goals  and  the  transformations  required  to  achieve  them.  The  goal  may  be  varied,  such  as  identifying  the 
proper  procedure,  assessing  the  status  of  back-up  systems,  or  diagnosing  a  problem  (Rasmussen,  1981).  To  achieve 
the  goals,  operators  generate  alternative  response  plans,  evaluate  them,  and  select  the  most  appropriate  one  that  is 
relevant  to  the  current  situation  model. 

While  this  is  the  basic  sequence  of  cognitive  activities  associated  with  response  planning,  one  or  more  of  these 
steps  may  be  skipped  or  modified  based  on  the  operator’s  assessment  of  a  particular  situation.  When  procedures 
are  available  and  judged  appropriate  to  the  current  situation,  the  need  to  generate  a  response  plan  in  real-time  may 
be  eliminated.  However,  even  when  written  procedures  are  available,  some  aspects  of  response  planning  will  be 
done.  For  example,  operators  still  need  to  (I)  identify  appropriate  goals  based  on  their  own  situation  assessment, 

(2)  select  the  appropriate  procedure,  (3)  evaluate  whether  the  procedure-defined  actions  are  sufficient  to  achieve 
those  goals,  and  (4)  adapt  the  procedure  to  the  situation,  if  necessary. 

The  decision  making  involved  in  situation  assessment  and  response  planning,  especially  in  ambiguous  situations, 
such  as  when  available  procedures  do  not  suffice,  can  be  a  large  cognitive  burden  and  draw  heavily  upon  working 
memory,  long-term  memory,  and  attentional  resources.  In  such  situations,  information  is  consciously  manipulated 
in  working  memory,  and  the  ability  to  do  so  is  a  direct  function  of  the  attentional  resources  available.  Working 
memory  has  very  limited  capacity,  and  without  sustained  attentional  resources  (or  transfer  of  the  information  to 
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long-term  memory),  information  decays  rapidly.  Information  can  be  lost  due  to  (1)  insufficient  attentional 
resources  to  keep  it  active,  (2)  overload  of  the  working  memory  capacity,  and  (3)  interference  from  other 
information  in  working  memory.  To  increase  the  capacity  of  working  memory,  operators  use  memory  heuristics, 
such  as  chunking,  that  enable  them  to  organize  various  bits  of  information  into  higher-level,  meaningful  units.  A 
heuristic,  as  used  in  this  report,  is  a  shortcut  for  information  processing  developed  through  experience  and  trial- 
and-error  rather  than  systematic,  formal  analysis.  Once  this  is  accomplished,  the  higher-level  units,  not  the 
individual  elements,  are  stored  in  working  memory. 

Roth,  Mumaw,  and  Lewis  (1994)  demonstrated  the  need  for  operators  to  maintain  a  supervisory  role  even  when 
responses  are  largely  dictated  by  EOPs.  They  investigated  how  operators  handle  cognitively  demanding 
emergencies.  Their  objective  was  to  examine  the  role  of  situation  assessment  and  response  planning  on  guiding 
crew’s  performance  in  situations  where  EOPs  were  being  used.  NPP  operators  from  two  different  utilities 
performed  interfacing  system  loss  of  coolant  accident  (ISLOCA)  and  loss  of  heat  sink  scenarios  on  training 
simulators,  where  complexities  made  it  difficult  to  simply  follow  the  appropriate  procedure.  The  results  showed 
the  importance  of  high-level  cognitive  functions  during  the  use  of  EOP.  The  operators  developed  an  understanding 
of  the  plant  state  and  confirmed  their  situation  assessment.  They  also  attempted  to  understand  plant  performance 
that  was  not  expected  based  on  their  current  situation  model.  These  cognitive  activities  enabled  them  to  evaluate 
the  appropriateness  of  the  EOP  for  the  high-level  goal  dictated  by  the  situation  assessment.  Roth  et  al.  (1994) 
noted  the  importance  of  the  crew’s  interaction  and  communication  to  these  high-level  cognitive  functions.  This 
was  partly  because  of  the  need  to  obtain  information  from  many  HSIs  in  different  locations.  In  addition, 
communication  helped  operators  overcome  the  fact  that  EOPs  do  not  address  all  the  important  information  about 
the  current  state  of  the  plant.  Roth  et  al.  noted  that  these  cognitive  activities  made  it  possible  for  crews  to  evaluate 
the  ability  of  a  procedure  to  achieve  its  high-level  goal  in  the  context  of  the  current  plant  condition.  When  a 
specific  procedure  failed  to  meet  the  high-level  goal,  operators  would  alter  its  path  to  better  address  the  situation. 

Thus,  Roth  et  al.  (1994)  demonstrated  the  importance  of  understanding  the  basis  of  the  procedure  and  the  higher- 
level  goals  it  is  intended  to  achieve.  The  need  to  formulate  modifications  to  procedure  pathways  also  means  that 
operators  may  not  simply  proceed  linearly  through  a  procedure.  They  may  need  to  consider  future  steps,  reexamine 
previous  steps  and  consult  other  procedures  to  verify  that  their  current  activities  are  correct  and  will  meet  the  high- 
level  goals  of  the  procedure. 

Roth  (1994)  considered  the  implication  of  the  Roth  et  al  (1994)  study  for  the  design  of  operator  support  systems. 
First,  the  requirement  of  situation  assessment  and  response  planning  independent  from  procedure  activity  suggests 
that  operators  must  maintain  awareness  of  abnormal  plant  symptoms,  determine  what  malfunctions  could  produce 
those  symptoms,  and  know  the  manual  and  system  actions  that  are  being  undertaken  and  their  effects.  Second, 
since  crews  have  to  anticipate  the  consequences  of  their  actions,  operator  support  systems  could  support  identifying 
consequences  and  side  effects  of  planned  actions.  Third,  operators  must  understand  the  assumptions  and  logic 
behind  the  procedures,  i.e.,  the  intent  of  the  procedures,  their  overall  strategies,  and  the  transition  logic  between 
them.  Since  operators  may  not  move  linearly  between  procedures,  the  means  for  navigating  the  information 
system  will  be  important  to  the  usefulness  of  such  information  systems  in  complex  emergencies. 

5.2.4  Response  Implementation 

Response  implementation  is  the  actual  performance  of  the  actions  identified  in  response  planning.  This  can  be  as 
simple  as  a  single  operator  selecting  and  operating  a  control,  or  it  can  involve  communications  and  coordination 
with  teams  of  operators  in  different  locations  of  the  plant,  who  each  then  select  and  operate  appropriate  equipment 
controls  in  a  centrally  coordinated  manner.  The  actions  may  be  discrete  (e.g.,  flipping  a  switch)  or  they  may 
involve  continuous  control  (e.g.,  controlling  steam  generator  level). 
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The  results  of  actions  are  monitored  through  feedback  loops.  Two  aspects  of  NPPs  can  make  response 
implementation  difficult:  response  time  and  indirect  observation.  Time  and  feedback  delays  disrupt  response 
implementation,  because  they  make  it  difficult  to  determine  whether  control  actions  are  having  their  intended 
effect.  In  such  a  situation,  the  operator’s  ability  to  predict  future  states  using  mental  models  can  be  more  important 
in  controlling  responses  than  feedback.  Further,  since  plant  processes  cannot  be  directly  observed,  their  status  is 
inferred  through  indications.  Thus,  errors  in  the  cognitive  process  can  disrupt  performance. 

Failures  in  response  implementation  can  lead  to  operating  the  wrong  equipment,  or  incorrectly  operating  or 
controlling  particular  components. 

5.2.5  Implications  For  Information  System  Design 

From  a  consideration  of  the  cognitive  tasks  performed  by  operators  to  support  their  supervisory  control  functions, 
the  following  general  observations  can  be  made: 

•  Operators’  information  needs,  especially  in  support  of  monitoring,  detection,  and  situation  assessment,  are 
very  broad,  both  in  terms  of  range  of  activities  they  need  to  be  aware  of,  and  the  level  of  abstraction  they  need 
to  consider. 

•  Information  should  be  expressed  in  terms  that  are  at  the  level  that  will  directly  impact  decision  making  and 
control.  Cognitive  resources  should  not  be  required  to  mentally  process  data  to  provide  the  information  that  is 
needed. 

•  It  is  important  to  correctly  map  the  plant  or  process  and  the  identification  of  information  needs  and  their 
relationship  to  the  operator’s  mental  model.  These  mappings  are  necessary  to  ensure  that  the  information 
representations  trigger  the  appropriate  mental  representation  of  the  situation. 

•  HS1  design  can  have  a  strong  effect  on  the  operator’s  cognitive  activity  and  can  make  information  difficult  to 
obtain  (through  poor  organization,  high  data  density,  and  excessive  interface  management  demands). 

The  following  information  processing  considerations  for  cognitive  tasks  were  reflected  in  design  principles 
supporting  situation  awareness  developed  by  Endsley  (1988,  1995): 

•  Displays  should  provide  information  that  is  processed  and  integrated  to  meet  the  operators’  need  to 
comprehend  the  current  situation  and  to  project  future  states. 

•  Information  presentation  should  take  into  account  the  operator’s  goals  and  the  information  needed  to  make 
goal-directed  decisions.  That  is,  information  should  be  organized  in  terms  of  what  the  operator  is  trying  to 
accomplish  and  not  simply  the  physical  layout  of  the  system. 

•  Since  situation  awareness  depends  on  activating  the  appropriate  mental  representation  for  the  situation,  the 
critical  cues  to  activating  the  schema  should  be  determined  and  displays  should  make  these  cues  salient. 

•  Cues  with  highly  salient  features  can  redirect  attention  away  from  the  current  goals.  Such  cues  should  be 
reserved  for  situations  that  require  the  redirection  of  goals  and  critical  events. 
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•  The  display  should  provide  global  situation  awareness;  that  is,  it  should  give  an  overview  of  the  status  of  all 
operator  goals  at  all  times  as  well  as  giving  details  about  the  current  goal.  This  is  because  situation  awareness 
suffers  when  operators  focus  on  some  information  and  fail  to  attend  to  other  important  information. 

•  Support  should  be  given  for  projecting  future  states  of  the  system  (especially  for  less  experienced  operators). 
For  example,  displays  of  trends  or  rate  of  change  support  this  process. 

•  Information  overload  should  be  avoided  by  filtering  out  extraneous  information  and  using  processing 
techniques  that  integrate  lower-level  data  to  provide  operators  with  the  high-level  information  needed  for  goal- 
directed  decision  making. 

•  To  minimize  divided  attention,  information  should  be  grouped  and  embedded  within  objects  when  possible. 
The  number  of  attention  shifts  should  be  minimized;  thus,  the  number  of  objects  should  be  minimized,  as 
should  the  need  to  shift  between  separate  displays.  Ideally,  all  information  should  be  available  at  a  single 
glance. 

•  The  HSI  should  support  parallel  processing  by  providing,  for  example,  information  through  visual  and 
auditory  channels.  This  will  help  operators  to  do  tasks  requiring  divided  attention. 

5.3  Analysis  of  Information  Requirements 

HFE  guidance  has  been  criticized  for  focusing  on  legibility  and  not  meaningfulness.  Bennett  et  al.  (1997)  stated 

that: 


Designs  that  consider  only  data  availability  often  impose  unnecessary  burdens  on  the  user-  to  collect  relevant  data,  to 
maintain  these  data  in  memory,  and  to  integrate  these  data  mentally  to  arrive  at  a  decision.  These  mental  activities 
require  extensive  knowledge,  tax  limited  resources  (attention  and  short-term  memory)  and  therefore  increase  the 
probability  of  poor  decision  making  and  errors,  (p.  673) 

However,  in  the  last  few  years,  the  NRC  has  improved  the  traditional  guidance  with  the  issuance  of  NUREG-071 1 
and  NUREG-0700,  Rev.  L  These  documents  contain  guidance  for  information  requirements  that  include  an 
operating  experience  review,  a  functional  requirements  analysis,  a  functional  allocation  analysis,  a  task  analysis, 
and  a  human  reliability  analysis.  Section  5.1  summarizes  these  documents. 

Further,  as  noted  in  Section  5.2,  Vicente  and  Rasmussen  (1992)  suggested  that  traditional  approaches  to 
information  system  design,  based  on  preplanned  evaluations  of  normal,  abnormal,  and  emergency  events,  will  fail 
to  meet  the  operators’  needs  in  the  face  of  events  that  have  not  been  planned  for  in  advance. 

Proponents  of  ecological  interface  design  (EID)  suggested  it  as  an  alternative  approach  to  traditional  HFE  design. 
The  EID  approach  has  implications  for  identifying  operator  information  needs  as  well  as  for  developing 
representations  to  display  plant  information.  The  former  is  addressed  in  this  section  and  the  latter  in  Section  5.4. 
The  intent  in  this  section  is  to  determine  whether  the  EID  approach  provides  alternative  or  additional  methods  that 
may  be  beneficial  in  the  information  requirements  area. 

5.3.1  The  EID  Approach  to  the  Analysis  of  Information  Requirements 

The  concept  of  an  ecological  approach  to  interface  design  can  be  traced  back  to  Gibson’s  theory  of  visual 
perception  (1979).  The  central  concept  in  Gibson’s  approach  is  that  human  behavior  cannot  be  understood  as  an 
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entity  independent  from  the  environment  in  which  it  occurs.  Thus,  knowledge  of  the  context  or  domain  within 
which  behavior  occurs  is  essential  to  understanding  human  performance.  Gibson  called  this  the  ecological 
approach  to  behavioral  analysis  (1979),  and  his  concepts  have  influenced  the  EID  model  of  interface  design.  The 
basis  for  using  ecological  perception  theory  as  an  approach  to  interface  design  was  described  in  several  papers 
(e.g.,  Flach,  1990;  Vicente  and  Rasmussen,  1990,  1992). 

In  the  nuclear  industry,  the  EID  approach  was  supported  by  Vicente  and  Rasmussen  (e.g.,  1992).  Rasmussen 
(1986)  viewed  human  performance  as  representing  behavior  organized  and  controlled  at  various  levels  of 
abstraction,  including  perception,  rules,  and  knowledge.  Operators  are  characterized  as  flexible  in  their  ability  to 
respond  to  the  situation.  This  characterization  of  action,  being  based  upon  various  levels  or  depths  of  processing, 
led  to  the  categorization  of  operator’s  performance  into  three  classes:  skill-based,  rule-based,  and  knowledge-based 
(SRK  scheme). 

Skill-based  actions  means  that  upon  perception  of  particular  signals,  the  operator  initiates  a  well- learned 
“over-trained”  response.  Pilots  are  trained  to  this  level  so  that  the  they  can  respond  quickly  without  the  need  for 
more  abstract  and  time-consuming  processing.  Behavior  in  this  context  is  largely  automatic,  initiated,  and 
controlled  outside  of  the  operator’s  awareness. 

Rule-based  performance  is  generally  more  complex  and  consciously  controlled,  yet  a  deep  level  of  processing  is  not 
required  since  the  situation  is  interpreted  as  familiar  and  recognized  as  requiring  a  previously  defined  set  of 
actions.  The  use  of  standardized  procedures,  on  which  an  operator  has  been  trained  in  response  to  a  well-defined 
transient,  reflects  primarily  rule-based  behavior.  In  Rasmussen’s  model,  most  of  the  linkages,  between  skill-based 
performance  and  the  “identification  of  a  state  of  ambiguity”  in  the  system,  fall  into  the  domain  of  rule-based 
performance. 

Knowledge-based  processing  represents  the  highest  level  of  abstraction  and  deepest  level  of  processing.  It  is 
invoked  when  neither  skill-based  nor  rule-based  responses  are  appropriate  to  the  situation  or  adequate  to  define  the 
actions  needed  to  meet  the  operator’s  goal.  The  situation  is  defined  as  unfamiliar  or  ambiguous,  and  operators 
must  develop  an  approach  to  attaining  goals  based  upon  their  in-depth  knowledge  of  the  plant  and  systems. 

Higher  levels  of  abstraction  enable  behavior  to  be  goal  directed.  These  levels  can  serve  to  define  goals  for  lower 
levels.  So,  for  example,  the  goal  to  define  the  present  state  of  the  system  can  structure  the  set  of  observations  made 
at  the  lower  level  of  abstraction.  When  a  target  or  desired  state  is  identified,  it  becomes  a  goal,  and  the  tasks  and 
procedures  to  be  used  to  achieve  that  state  are  defined.  As  noted  in  Section  5.2,  the  importance  of  the  goal-directed 
structure  of  information  processing  has  been  widely  recognized  inNPP  operator  modeling  (e.g.,  Bainbridge,  1974, 
1986).  The  influence  of  higher-level  goal  states  and  expectations  on  lower-level  cognitive  processes  also 
recognized  in  basic  cognitive  research  as  well,  e.g.,  the  effects  of  goal  states  on  pattern  recognition  (e.g.,  Neisser, 
1976). 

In  designing  HSls  for  these  types  of  behavior,  the  issue  is  to  determine  the  information  necessary  to  support  them 
at  the  various  levels  of  abstraction.  Based  on  his  observations  of  diagnosis  and  problem-solving  behavior, 
Rasmussen  (1986)  noted  that  information  about  a  system  varied  along  two  dimensions:  abstraction  and 
aggregation.  Abstraction  reflects  the  conceptualization  of  system  means-ends  relationships  along  a  hierarchy 
consisting  of  five  levels  from  physical  form  to  functional  purpose.  These  levels  were  defined  as  follows: 

•  Functional  purpose  -  the  purpose  for  which  the  system  was  designed 

•  Abstract  function  -  the  causal  structure  of  the  process  in  terms  of  mass,  energy',  information  or  value  flows 
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Table  5.  1  Abstraction-Aggregation  (A-A)  Matrix 


Level  of 
Abstraction 

Plant 

Level  of  Ag 

System 

[gregation1 

Equipment 

Component 

Functional  Purpose 

Abstract  Function 

Generalized  Function 

Physical  Function 

Physical  Form 

1  Levels  of  aggregation  as  typical  for  a  nuclear  power  plant,  but  they  are  examples  only. 


•  Generalized  function  -  the  basic  functions  a  system  was  designed  to  achieve 

•  Physical  function  -  the  characteristics  of  the  components  and  their  interconnections 

•  Physical  form  -  the  appearance  and  spatial  location  of  those  components. 

These  levels  span  the  physical  components  in  the  system  and  its  overall  functional  purpose.  Since  SRK  behavior 
requires  information  about  the  plant  at  various  levels,  the  abstraction  hierarchy  provides  a  framework  within  which 
to  describe  those  needs  and  consider  display  design. 

Aggregation  reflects  the  degree  of  integration,  from  part  to  whole,  that  problem-solving  performance  is  directed 
toward.  The  part-whole  dimension  also  consists  of  levels  ranging  from  the  individual  component  to  the  entire 
system.  These  two  dimensions  form  a  cognitive  matrix  describing  the  information  needs  of  plant  personnel,  called 
the  abstraction-aggregation  (A-A)  matrix  (see  Table  5.1). 

The  problem  solver  shifts  between  levels  of  abstraction  and  aggregation  to  solve  problems.  This  enables  the 
problem  to  be  conceptualized  in  different  ways.  For  example,  Rasmussen  (1986)  mapped  the  strategy  used  by  a 
computer  analyst  during  troubleshooting  into  this  problem-solving  space  (Figure  5.2).  Itoh  et  al.  (1995)  applied 
this  two-dimensional  concept  to  interface  design  for  nuclear  plants  (Figure  5.3). 

Thus,  during  diagnostic  performance,  the  information  needs  of  the  operator  vary  within  this  problem  space. 
However,  the  two  dimensions  tend  to  be  coupled  (Vicente,  1992);  that  is,  one  tends  to  think  at  lower  levels  of 
abstraction  when  dealing  with  components,  and  to  think  functionally  when  working  at  the  plant  level.  Lower 
levels  of  abstraction  support  knowing  what  to  do  and  higher  levels  support  an  understanding  of  why  it  is  necessary. 

In  contrast  to  the  information  needs  reflected  by  the  A-A  matrix,  Rasmussen  (1986)  suggested  that  in  many 
industrial  complexes,  HSIs  provide  only  one  level  of  abstraction,  reflecting  a  ‘one  sensor  -  one  indicator’  approach. 
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Figure  5.2  Diagnostic  Strategies  Used  in  Troubleshooting 

(Source:  Reprinted  with  the  author’s  permission  from  Rasmussen,  J  ,  “Information  Processing  and 
Human-Machine  Interaction,”  North  Holland,  New  York,  NY,  1986  All  rights  reserved  by  the  author.) 

This  makes  tasks  complex  because  the  data  available  often  do  not  match  the  information  needed.  Rasmussen  noted 
that 


In  the  design  of  human-machine  interfaces  for  supervisory  control  as,  e.g.,  industrial  process  control  consoles,  the 
emphasis  has  traditionally  been  on  the  presentation  of  measured  data  representing  the  physical  state  of  the  system  and 
its  processes.  In  complex  systems,  this  information  has  been  supplemented  by  information  about  the  underlying 
functional  structure  by  graphical  means  such  as  mimic  diagrams,  etc.  This  information  is  intended  to  serve  the 
controller’s  identification  of  the  actual  state  of  processes  bottom-up  through  the  hierarchy.  Information  representing 
the  intentions  behind  the  system  design,  in  terms  of  the  purpose  of  functions  and  equipment  and  of  constraints  upon 
the  acceptable  operation,  for  instance,  as  derived  from  safety  considerations,  has  only  been  very  sparsely  represented. 
This  means  that  the  interface  system  gives  little  or  no  support  in  the  top-down  derivation  of  the  proper  or  acceptable 
states  of  processes.  This  kind  of  information  was  supposed  to  be  immediately  available  to  operators  from  their  basic 
training.  However,  as  systems  become  complex  and  potentially  risky...  this  can  no  longer  be  assumed.  During  such 
situations,  information  about  the  purpose  of  interlocks,  properties  of  equipment  if  used  for  untraditional  purposes, 
etc.,  may  be  vital  for  ad  hoc  improvisations.  Consequently,  it  becomes  increasingly  important  to  include  in  the 
support  for  decision  making  the  information  needed  for  top-down  consideration  of  reasons.  This  means  that  the 
properties  of  the  system  to  be  controlled  should  be  represented  in  terms  of  a  consistent  means-end  hierarchy, 
systematically  mapping  the  purpose/function/equipment  relationships,  (pp.  130-131) 
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Figure  5.3  Electrical  Production  Abstraction  Hierarchy 

(Source:  Reprinted  from  Itoh,  J.,  Sakuma,  A.  and  Monta,  K.,  “An  Ecological  Interface 
for  Supervisory  Control  of  BWR  Nuclear  Power  Plants,”  Control  Engineering  Practice, 
3(2),  1995,  Copyright  1995,  with  permission  from  Elsevier  Science.) 
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Rasmussen  indicated  that  this  situation  places  the  burden  on  operators  to  find  the  information  they  need  to  support 
their  current  activities  and  to  structure  the  representation  of  the  system  that  best  meets  their  information  needs. 

The  advantage  of  defining  information  needs  using  the  A-A  matrix  is  that  normal  plant  behavior  is  dictated  by  a 
set  of  known  relationships  between  plant  parameters  governed  by  constraints  dictated  by  the  physical  nature  of 
plant  equipment,  the  physical  laws  governing  the  process,  and  the  purposes  for  which  the  system  was  designed. 
Faults  are  revealed  when  these  constraints  are  violated.  By  analyzing  information  needs  according  to  the  A-A 
matrix,  these  constraints  can  be  represented  in  the  design  of  the  information  system. 

Vicente  (1997)  suggested  that  process  control  operators  exist  in  a  correspondence-driven  domain  where  there  is  an 
external  reality.  This  concept  is  in  contrast  to  a  coherence-driven  domain  where  the  human  and  the  computer 
system  represent  a  closed  world  and  do  not  need  to  reflect  external  constraints.  The  information  system  should 
reflect  the  actual  constraints  of  the  process  being  controlled,  Vicente  refers  to  this  as  the  “ecological  compatibility 
principle”  (in  contrast  to  the  proximity-compatibility  principle,  discussed  in  Section  5.4). 

According  to  Vicente  and  Rasmussen  (1988,  1992),  the  general  goal  of  ecological  interface  design  is  to  “...design 
interfaces  that  do  not  force  cognitive  control  to  a  higher  level  than  the  demands  of  the  task  require,  but  that  also 
provide  the  appropriate  support  for  all  three  levels”  (p.  598).  The  following  three  general  principles,  related  to 
skill-,  rule-,  and  knowledge-based  behavior,  are  proposed  to  achieve  this  goal: 

•  Skill-based  behavior  -  “To  support  interaction  via  time-space  signals,  the  operator  should  be  able  to  act 
directly  on  the  display,  and  the  structure  of  the  displayed  information  should  be  isomorphic  to  the  part-whole 
structure  of  movements”  (Vicente  and  Rasmussen,  1992,  p.  598).  This  means  that,  as  far  as  possible,  the 
interface  should  give  the  information  necessary  to  allow  for  the  appropriate  use  of  skill-based  behavior,  which 
is  the  least  cognitively  demanding.  Designs  that  use  the  natural  tendencies  of  the  user  support  this  objective 
(i.e.,  direct-manipulation  interfaces). 

•  Rule-based  behavior  -  “Provide  a  consistent  one-to-one  mapping  between  the  work  domain  constraints  and 
the  cues  or  signs  provided  by  the  interface”  (Vicente  and  Rasmussen,  1992,  p.  598).  That  is,  provide  a 
mapping  between  the  invariant  properties  of  the  process  and  the  format  of  the  HS1  This  principle  is  supported 
by  showing  contextual  information  in  a  display,  such  as  stating  the  limitations  of  displayed  information.  This 
allows  operators  to  identify  potential  inadequacies  of  typical  responses  that  may  be  inappropriate  to  the  current 
situation,  thus  avoiding  procedural  traps  (Vicente,  1991). 

•  Knowledge-based  behavior  -  “Represent  the  work  domain  in  the  form  of  an  abstraction  hierarchy  to  serve  as 
an  externalized  mental  model  that  will  support  knowledge-based  problem  solving”  (Vicente  and  Rasmussen, 

1 992,  p.  599),  That  is,  display  the  relational  structures  of  the  processes  as  a  functional  hierarchy.  This 
principle  is  supported  by  providing  information  in  familiar  representations,  such  as  displaying  functional 
relationships  and  information  on  parameters  as  P&IDs  and  in  abstractions,  such  as  subcooling  margin. 

In  the  next  section,  we  address  the  effectiveness  of  the  EID  approach  in  the  analysis  of  information  requirements. 

5.3.2  Research  on  the  Effectiveness  of  EID  Displays 

5.3.2. 1  DURESS  EID  Display 

Vicente  and  colleagues  conducted  several  studies  to  assess  the  value  of  the  EID  approach  to  display  design. 

Vicente  (1991 )  undertook  the  studies  in  part  because  “...no  experiment  has  ever  compared  the  performance  of  a 
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multilevel  interface  based  on  the  abstraction  hierarchy  with  any  other  type  of  interface”  (p.  65).  One  series  of 
Vicente’s  studies  used  a  thermal-hydraulic  process  simulation  called  DURESS  (DUal  REservoir  System 
Simulation)  and  DURESS  II,  with  an  updated  simulation  model. 

The  DURESS  model  has  two  redundant  feedwater  streams,  which  feed  two  reservoirs  under  various  configurations 
The  operator  tries  to  keep  each  reservoir  at  a  specific  temperature  and  to  maintain  water  levels  as  needed  based  on 
flow  rates.  The  operator  accomplishes  this  task  by  controlling  valves,  pumps,  and  heaters.  The  simulation  is 
dynamic  with  time  lags  on  control  variables.  Faults  can  be  introduced  into  the  system  and  equipment  can  be 
damaged  by  improper  operations  (such  as  allowing  the  water  level  to  get  too  low  in  a  reservoir  while  it  is  heating). 

Operators  controlled  the  simulation  using  one  of  two  interfaces:  a  basic  physical  representation  (P)  of  the  process, 
or  a  physical  plus  a  functional  display  (P+F).  Figures  5.4  and  5.5  show  the  interfaces.  The  P  interface  was  a 
mimic-type  display  showing  two  feedwater  systems  (A  and  B),  their  components  and  status  such  as:  P  (Pump  on  or 
oft),  V  (valve  position),  HTR  (heating  rates)  for  reservoirs,  D  (demand),  and  T  (temperature).  In  addition  to  the 
representation  of  system  components  and  parameters,  the  P+F  display  contained  functional  information  identified 
by  analyzing  the  abstraction  hierarchy.  That  is,  the  display  included  information  on  the  status  of  the  functions  that 
the  components  were  serving  and  the  mass-balance  display  (see  the  right  side  Figure  5.6). 

The  DURESS  testbed  provided  an  opportunity  to  experimentally  study  display  effects  in  the  laboratory  with  a 
simplified  simulation  that  has  many  characteristics  similar  to  actual  process  control  systems.  The  simulation  and 
the  interfaces  are  described  in  detail  in  Vicente  (1991)  and  in  many  of  the  other  Vicente  references  discussed  in 
this  section.  Collectively,  these  studies  examine  the  effects  of  the  following:  underlying  design  philosophy  on 
performance,  varying  the  information  content  of  displays,  expertise  on  performance,  displays  on  mental  models, 
and  displays  on  normal  operation  and  fault  management  performance  and  strategies. 

Before  examining  the  results  of  these  studies,  we  will  examine  the  basis  for  the  two  interfaces  in  more  detail.  This 
is  important  because  the  studies  use  P  (traditional  design)  vs.  P+F  (EID  design)  as  the  principal  independent 
variable.  Therefore,  general  conclusions  about  the  strengths  and  weaknesses  of  EID  are  based  on  the 
representativeness  of  these  designs.  The  design  of  the  interfaces  is  discussed  in  detail  in  Chapter  IV  of  Vicente 
(1991).  To  briefly  summarize,  the  EID  process  first  requires  representing  the  domain  constraints.  That  is,  the 
elements  of  the  system  are  described  in  terms  of  the  abstraction  hierarchy.  Thus,  for  example,  the  functional 
purpose  (high-level  objective)  of  a  reservoir  is  to  maintain  sufficient  water  to  satisfy  output  demand  and  to  keep  the 
water  at  a  temperature  criterion.  The  next  step,  building  an  interface  that  represents  the  abstraction  hierarchy,  was 
accomplished  by  mapping  this  information  into  visible  geometric  properties  of  the  objects  in  the  interface.  This  is 
how  the  representation  is  thought  to  lessen  the  operator’s  workload  by  providing  an  external,  visible  model  of  the 
system.  This  latter  step  is  not  actually  a  function  of  EID;  EID  provides  the  requirements  for  the  information  that 
needs  to  be  represented,  not  how  to  represent  it.  The  following  elements  in  the  P+F  interface  (see  Figure  5.6) 
reflect  the  result  of  this  design  process: 

•  Functional  purpose  -  Demand  values  D1  and  D2  and  temperatures  T1  and  T2 

•  Abstract  function  -  the  rectangular  graphic  (to  the  right)  representing  mass  balance  of  the  reservoir 

•  Generalized  function  -  flowrates  for  feedwater  streams,  such  as  FA1  and  FVA 

•  Physical  function  -  valve  settings,  such  as  VB 

•  Physical  form  -  the  spatial  layout  of  the  components  and  their  connections 
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Figure  5.4  Physical  Interface  (P)  for  DURESS 

(Source:  Reprinted  with  the  author’s  permission  from  Vicente,  K.,  “Supporting  Knowledge-Based 
Behavior  Through  Ecological  Interface  Design,”  (EPRL-91-01),  University  of  Illinois,  Engineering 
Psychology  Research  Laboratory,  Urbana,  IL,  1991.) 


The  P+F  EID  interface  appears  to  be  a  well  thought-out  example  of  an  EID  process. 

The  design  basis  for  the  P  interface,  which  is  a  traditional  interface,  is  not  discussed.  However,  it  is  not  clear  that 
the  P  interface  is  the  product  of  a  design  approach  that  is  an  alternative  to  the  EID  process.  Thus,  one  cannot 
determine  why  basic  parameter  values  that  appear  in  the  P+F  interface  do  not  appear  in  the  P  interface.  This  latter 
point  was  discussed  in  an  interesting  exchange  in  Human  Factors  addressing  a  critique  of  Christoffersen,  Hunter, 
and  Vicente  (1996)  by  Maddox  (1996)  and  Vicente,  Christoffersen,  and  Hunter  (1996).  The  lack  of  information  on 
flow  rate  either  from  the  pump  or  to  the  reservoirs  is  noteworthy.  Only  information  on  valve  position  is  given. 
Typical  process  displays  would  more  likely  include  information  on  flow  rate  rather  than  valve  position,  and  flow 
rate  should  be  more  useful  for  process  control.  If  the  P  interface  did  not  reflect  a  systems  analysis  or  even  a 
“traditional”  task  analysis,  then  what  was  used  to  determine  the  information  requirements  of  the  P  interface?  This 
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Figure  5.5  Physical  +  Functional  (P+F)  Interface  for  DURESS 
(Source:  Reprinted  with  the  author’s  permission  from  Vicente,  K.,  “Supporting  Knowledge-Based 
Behavior  Through  Ecological  Interface  Design,”  (EPRL-91-01),  University  of  Illinois,  Engineering 
Psychology  Research  Laboratory,  Urbana,  1L,  1991.) 


is  an  important  question;  its  answer  affects  any  conclusions  on  the  generalizability  of  identified  differences  in 
performance  between  the  P  and  the  P+F  displays. 

With  this  issue  in  mind,  the  research  related  to  the  EID  DURESS  interface  will  be  discussed.  Seven  different 
experiments  of  the  series  are  discussed  here. 

The  first  experimental  evaluation  of  the  interfaces  is  reported  in  Vicente  (1992)  (and  Experiment  1  of  Vicente, 
Christo ffersen,  and  Pereklita,  1995).  The  evaluation  compared  the  two  interfaces  on  fault  diagnosis  accuracy  and  a 
recall  memory  measure.  A  second  independent  variable  was  expertise  (expert  and  novice).  Experts  were  graduate 
students  in  mechanical  or  nuclear  engineering,  while  the  novices  were  graduate  students  who  had  never  been 
enrolled  in  a  science  or  engineering  major.  Subjects  were  first  provided  with  familiarization  and  training  on  the 
simulator  and  interface.  There  were  12  participants  in  each  expertise  group.  Participants  observed  the  DURESS 
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simulation  for  approximately  30  seconds.  In  some  of  the  trials,  faults,  such  as  reservoir  leak  and  a  blocked  valve, 
occurred.  Following  the  presentation,  the  screen  went  blank,  and  the  participants  were  asked  to  recall  the  value  of 
system  parameters  and  diagnose  the  functional  state  of  the  system. 

The  results  showed  that  the  P+F  interface  was  better  than  the  P  display  in  support  of  diagnosis.  Generally,  this 
result  was  stronger  for  experts  than  for  novices.  Vicente  (1992)  indicates  that  the  superiority  of  diagnosis  using  the 
P+F  interface  supports  the  value  of  including  higher-level  functional  information  in  a  display,  which  he  believes  is 
critical  to  diagnosis.  On  the  memory  measures,  experts  had  better  recall  than  did  novices;  however,  the  effect  of 
recall  using  the  P+F  display  was  only  better  for  functional  information.  (The  memory  results  were  reported  in 
more  detail  in  Vicente  [1992].  It  was  reported  that  memory  for  the  subset  of  variables  that  were  most  important  to 
the  diagnosis  was  better  for  the  participants  using  the  P+F  display.)  Vicente  also  concluded  that  the  study  provided 
empirical  support  for  displaying  various  levels  of  the  abstraction  hierarchy  in  an  interface.  That  is,  he  concluded 
that  the  results  are  not  due  simply  to  the  fact  that  the  P+F  display  has  more  information  than  the  P  display,  but  that 
the  additional  information  was  specified  by  the  EID  approach  as  being  goal  relevant. 

In  the  second  study,  Pawlak  and  Vicente  (1996)  compared  the  P  and  P+F  displays  to  determine  their  relative  effects 
on  (1)  control  strategies  and  (2)  fault  detection  and  diagnosis.  They  hypothesized  that  the  EID  display  would  lead 
to  the  development  of  more  effective  control  strategies.  Twelve  participants  were  assigned  either  to  the  P  or  the 
P+F  display  (six  each).  They  practiced  for  an  hour  a  day  for  27  days  and  performed  the  following  tasks:  start-up, 
tuning  to  new  setpoints,  shutdown,  and  fault  management.  For  some  of  the  trials,  participants  had  to  conduct 
secondary  tasks  along  with  their  primary  tasks.  The  secondary  task  loaded  on  either  verbal  or  spatial  cognitive 
resources.  Measures  of  primary  task  performance  were  time  and  analysis  of  verbal  protocols. 

The  results  showed  that  the  P+F  display  led  to  more  accurate  diagnosis  than  the  P  display,  although  the  time  to 
compensate  for  the  fault  did  not  differ  between  the  two  groups  (compensating  for  a  fault  does  not  necessarily 
require  a  diagnosis).  Further,  the  strategies  used  by  the  two  groups  were  different.  Although  some  differences  in 
strategy  on  the  startup  task  were  observed,  no  significant  differences  in  general  task  performance  were  found. 
Analyses  of  the  workload  measures  showed  that  the  P+F  display  loaded  more  on  spatial  cognitive  resources,  while 
the  P  interface  loaded  more  on  verbal  resources. 

The  authors  identified  some  limitations  to  the  study.  First,  it  is  not  known  whether  differences  in  diagnostic 
performance  were  due  to  differences  in  the  content  or  in  the  form  of  the  displays.  (This  is  a  key  point  relevant  to 
interface  design,  because  the  added  information  could  easily  be  given  in  a  more  standard  format,  resulting  in  a  very 
different  display.)  Second,  the  participants  were  not  skilled  in  detecting  faults  and  it  is  not  known  whether  the 
benefits  of  the  EID  interface  would  hold  for  true  operational  experts. 

In  the  third  study,  EID  displays  were  compared  with  P&ID  displays  in  a  longitudinal  examination  of  the 
performance  of  a  variety  of  simulated  process  control  tasks  (Christoffersen  et  al.,  1994,  1995,  1996).  The  DURESS 
II  simulation  was  used  and  the  P  and  P+F  displays  were  compared.  Six  students  participated  in  the  study.  They 
were  assigned  to  either  the  P  or  the  P+F  condition  in  a  between-subjects  design  and  did  the  same  operational  tasks 
as  described  above.  The  participants  engaged  in  224  trials  over  6  months.  A  brief  crossover  was  performed  where 
participants  used  the  other  group’s  displays.  The  primary  performance  measures  were  trial  completion  time  and 
verbal  protocols. 

The  most  significant  finding  was  that  the  EID  display  led  to  faster  fault  detection,  more  accurate  fault  diagnosis, 
and  faster  compensation.  These  differences  were  observed  for  routine  as  well  as  non-routine  faults  (Christoffersen 
et  al.,  1995).  Also  significant  is  that  their  strategies  were  different  than  the  P  group.  The  P  group  had  difficulty 
distinguishing  between  a  fault  and  a  normal  situation  requiring  control  action.  They  sometimes  only  recognized  a 
fault  after  normal  control  actions  failed.  They  typically  did  not  diagnose  the  failure  and  handled  it  by  trial  and 
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error.  In  contrast,  P+F  participants  typically  did  not  adopt  such  a  strategy.  Instead,  they  were  often  able  to 
diagnose  the  problem,  and  therefore,  respond  appropriately. 

For  performance  on  a  normal  (nondisturbance)  trial,  there  was  very  little  difference  in  mean  performance  between 
the  two  groups.  However,  the  P+F  group  was  significantly  less  variable  than  the  P  group.  Interestingly,  during  the 
crossover  trials,  when  participants  in  the  P+F  group  used  the  P  interface,  their  performance  variability  increased. 
Thus,  the  differences  were  likely  due  to  the  characteristics  of  the  interface  and  not  the  characteristics  of  the 
participants  alone.  This  finding  is  important  because  assuming  acceptable  mean  performance,  it  is  important  from 
a  human  reliability  perspective  to  reduce  human  variability. 

In  the  fourth  study,  the  importance  of  the  functional  level  of  information  was  examined  by  having  participants 
control  the  DURESS  II  process  using  a  windows-based  interface  (Janzen  and  Vicente,  1997).  The  windows 
presented  distinct  information  reflecting  four  of  the  five  levels  of  the  abstraction  hierarchy:  goals,  principles,  flow, 
and  settings.  No  form  level  was  included  because  the  process  was  simulated.  Only  one  level  could  be  observed  at  a 
time.  Of  interest  were  which  levels  were  used,  for  what  purposes,  and  with  what  results.  Six  participants  in  three 
types  of  trails  (normal,  routine  fault,  and  non-routine  fault)  operated  the  simulation  for  approximately  one  hour  per 
workday  for  one  month.  The  performance  measures  included  system  variables,  operator  actions,  and  verbal 
protocols.  Regression  analyses  were  conducted  to  predict  process  control  performance,  based  on  the  participant’s 
attention  allocation  strategies. 

The  results  showed  that  more  accurate  and  faster  control  under  normal  conditions  was  achieved  when  more 
functional  levels  were  consulted  (relative  frequency  of  visits  to  the  principles  level).  In  addition,  while  fault 
detection  time  was  not  predicted,  allocating  attention  to  more  levels  (relative  frequency  of  visits  to  the  principles 
and  flows  levels)  was  associated  with  more  accurate  diagnosis.  For  fault  compensation,  performance  was  improved 
by  dwell  time  in  the  principles  level.  The  authors  noted  that  to  simply  complete  the  process  control  task,  only  the 
settings  and  goals  levels  were  necessary.  To  complete  it  quickly  and  stably,  the  principles  level  is  necessary. 
Further,  to  handle  faults,  moving  between  multiple  levels  is  required;  i.e.,  most  faults  are  revealed  by  switching 
between  the  settings  and  flows  levels  or  between  the  flows  and  principles  levels.  These  results  showed  the 
importance  of  the  different  levels  of  the  abstraction  hierarchy  to  different  operating  tasks. 

In  the  fifth  study,  Vicente  and  Wang  (1996)  extended  the  comparison  of  effects  of  EID  performance  by  comparing 
the  following  five  interfaces,  which  included  those  previously  described  and  some  variations  on  them: 

•  the  P  display, 

•  the  P  display+trend  plots  (for  input  flow  rates,  level,  and  output  flow  rates  for  each  reservoir), 

•  the  P+F  display, 

•  the  P+F  display+trend  plots,  and 

•  the  P+F  display+analytical  redundance. 

Analytical  redundance  is  the  calculation  of  expected  parameter  values  using  a  model  of  system  performance.  The 
expected  value  is  then  represented  in  the  display  along  with  the  actual  value.  Deviations  between  the  two  show 
some  disturbance  or  abnormality  of  the  system  (Figure  5.6).  The  difference  between  the  actual  and  expected  flow 
rate  is  represented  by  the  difference  between  the  solid  and  dashed  lines  connecting  MI  and  MO,  which  means  a 
leak  in  the  reservoir.  Such  a  display  directly  reveals  the  violations  of  process  constraints  introduced  by  the  fault. 
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Figure  5.6  EID  P+F  Interface  with  Analytical  Redundancy 

(Source:  Reprinted  with  permission  from  Vicente,  K.,  and  Wang,  J.,  “Taking 
Full  Advantage  of  Process  Constraints  in  Advanced  Interface  Design,” 
Proceedings  of  the  J996  American  Nuclear  Society  International  Topical 
Meeting  on  Nuclear  Power  Plant  Instrumentation,  Control  and  Human- 
Machine  Interface  Technologies ,  pp.  405-41 1,  1996.  Copyright  1996  by  the 
American  Nuclear  Society  .  All  rights  reserved.) 


Participants  operated  DURESS  under  startup  tuning  and  fault  management  trials.  Performance  measures  included 
task  times,  fault  detection  time  and  accuracy,  and  verbal  reports.  The  fault  detection  time  was  better  for  the 
EID+analytical  redundancy,  but  only  for  reservoir  leaks,  as  expected.  They  were  better  at  fault  compensation  as 
well.  For  diagnosis,  the  three  EID  groups  were  more  accurate  than  the  P  groups.  Including  trend  plots  did  not 
clearly  improve  fault  management,  and  in  some  cases  even  decreased  performance. 


In  the  sixth  study,  Howie  and  Vicente  (1996)  followed  the  expertise  developed  by  six  novice  operators  of  the 
DURESS  II  process  simulation.  They  anticipated  greater  expertise  would  develop  in  a  group  using  an  EID 
interface.  Half  were  assigned  to  a  P  interface  group  and  half  to  a  P+F  interface  group.  Perhaps  the  most 
interesting  aspect  of  this  study  is  the  dependent  measures  developed  for  analyzing  performance:  steady-state  times, 
complexity  of  action  transition  graph,  path  length  of  state  space,  mass-energy  inventories,  and  timelines.  The 
operators  completed  224  trials  over  6  months,  and  the  first  20  trials  were  compared  with  the  last  20  trials. 

Performance  improvements  of  larger  magnitude  were  reported  for  the  EID  group,  which  may  be  because  they 
tended  to  perform  more  poorly  in  the  first  20  trials.  By  the  last  20,  both  groups  performed  similarly.  This  pattern 
held  for  the  first  three  measures.  As  an  example,  Table  5.2  shows  the  average  steady-state  times.  Faster  times 
show  an  ability  to  reach  goals  quickly  and  to  maintain  the  system  in  the  goal  state.  There  may  be  two  reasons  for 
this  result.  First,  with  only  three  participants  per  group,  there  is  no  reason  to  believe  the  groups  were  equivalent  at 
the  start  of  the  study.  Perhaps  by  chance,  the  EID  group  performed  more  poorly  at  the  task  in  the  beginning. 
Second,  the  EID  group  may  have  been  harder  for  the  participants  to  use  during  early  trials  so  they  performed 
poorer.  Over  time  they  may  have  learned  to  use  the  EID  display  and  improved.  If  the  latter  interpretation  was 
correct,  it  would  not  represent  an  especially  positive  effect  of  EID  displays.  For  the  mass-energy  inventory,  the 
P+F  group  performed  better  for  one  of  the  two  reservoirs  in  terms  of  percent  above  1/3  capacity. 
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Table  5.  2  Mean  Steady-State  Times  (in  seconds) 


Initial  Trials 

Final  Trials 

P+F  Group 

687.6 

395.1 

P  Group 

602.2 

393.3 

In  the  seventh  study,  Vicente,  Christoffersen,  and  Pereklita  (1995)  evaluated  the  hypothesis  that  superior  fault 
management  performance  with  EID  interfaces  occurs  because  these  interfaces  encourage  knowledge-based 
behavior  by  enabling  operators  to  (1)  begin  diagnosis  at  high  levels  of  abstraction,  reflecting  functional 
considerations,  and  (2)  use  that  information  to  work  their  way  down  to  problematic  components  at  the  lower  levels 
of  abstraction.  To  evaluate  this  hypothesis,  fault  diagnosis  strategies  were  examined  as  a  function  of  type  of 
expertise  and  success  at  fault  management  It  was  found  that  the  relevant  expertise  in  managing  DURESS  faults 
was  knowledge  of  the  DURESS  system  and  the  P+F  interface.  Experience  at  controlling  DURESS  and 
thermodynamic  knowledge  were  not  well  correlated  with  performance.  For  strategies  of  fault  management,  better 
performers  began  at  higher  levels  of  abstraction  and  slowly  worked  their  way  down  to  lower  levels.  In  fact, 
diagnosis  strategies  starting  at  the  lower  levels  had  a  negative  effect  on  performance.  In  part,  this  is  because  at 
lower  levels  components  have  many  interconnections.  Thus,  it  is  difficult  to  determine  where  a  problem  resides. 

In  summary,  the  main  result  of  these  studies  is  that  fault  detection  and  diagnosis  seemed  to  be  improved  with  the 
EID  display  reflecting  the  abstraction  hierarchy.  There  was  also  a  reduction  in  control  performance  variability. 
However,  several  issues  make  it  difficult  to  fully  interpret  the  results. 

The  first  issue  is  generalization  of  the  findings  as  representative  of  the  strength  and  weaknesses  of  the  design 
approaches  they  intended  to  represent.  The  uncertain  technical  basis  for  the  P  display  make  it  unclear  whether  the 
results  reflect  the  contribution  of  EID  over  traditional  design  methods,  or  whether  they  reflect  the  difference 
between  a  display  that  was  designed  with  operator  information  needs  defined  and  one  that  was  not.  Or,  perhaps 
more  likely,  all  three  aspects  of  the  EID  displays  (added  information  about  flow  rate,  improved  display  design,  and 
analytical  redundancy)  created  improvements  in  performance,  but  how  much  each  aspect  contributed  or  which  one 
is  more  important  cannot  be  determined. 

The  second  issue  is  generalization  of  the  findings  to  more  complex  systems.  DURESS  is  a  relatively  simple  system 
compared  to  a  NPP.  Thus,  all  the  levels  of  the  abstraction  hierarchy  could  be  represented  in  a  single  display, 
which  would  not  be  possible  in  an  actual  NPP.  Consequently,  the  amount  of  information  (note  the  difference  in 
amount  of  information  between  the  P  and  P+F  interfaces  and  generalize  to  an  entire  plant)  and  its  organization 
(with  regard  to  functions,  systems,  and  levels  of  the  hierarchy)  may  be  an  issue. 

The  third  issue  is  generalization  of  the  findings  to  more  complex  and  routine  tasks.  The  main  benefits  of  EID  were 
seen  in  knowledge-based  diagnosis.  In  an  actual  plant,  this  cognitive  activity  is  supported  by  alarm  systems  and 
other  HSIs  and  procedures.  The  support  of  EID  displays  in  the  context  of  a  richer  HSI  and  task  environment  needs 
to  be  addressed.  Further,  while  the  effects  on  diagnostic  performance  are  important,  a  NPP  information  system 
must  support  all  of  the  cognitive  activities  discussed  in  Section  5.2.  Since  most  of  the  activities  are  highly 
proceduralized,  including  responses  to  planned  events  and  EOPs,  using  displays  based  on  EID  for  these  tasks 
should  be  evaluated. 
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The  fourth  issue  is  generalization  of  the  findings  to  other  participants,  especially  expert  operators.  Some  of  the 
studies  were  based  on  very  small  sample  sizes,  and  therefore,  the  results  may  be  influenced  by  individual 
differences  (Howie  and  Vicente,  1996).  Even  when  statistical  tests  are  conducted  to  determine  the  significance  of 
observed  differences,  there  is  no  guarantee  of  the  generalizability  of  results  from  small  samples,  because 
generalization  is  based  on  the  assumption  that  randomization  has  resulted  in  equivalent  groups.  The  confounding 
effects  of  individual  differences  are  minimized  by  randomization  of  participants  to  groups.  When  accomplished  in 
large  numbers,  the  observed  differences  between  groups  can  be  attributed  to  the  independent  variable.  When  this  is 
not  accomplished,  the  results  may  be  due  to  the  manipulations  of  the  independent  variable  (e.g.,  EID  vs.  traditional 
display)  or  to  preexisting  differences  in  ability  between  the  groups  (which  is  not  accounted  for  in  the  statistical 
tests).  Further,  generalization  to  significant  groups  not  represented  in  the  studies  are  not  supported.  In  this  case, 
the  effects  on  expert  operators  are  unknown,  and  (as  discussed  below)  they  may  interact  differently  with  the 
displays  due  to  their  expertise. 

The  fifth  issue  is  identification  of  an  appropriate  mental  model.  Vicente  concluded  that  “an  interface  based  on  an 
abstraction  hierarchy  can  provide  more  support  for  knowledge-based  behavior  than  an  interface  based  on  physical 
variables  alone  because  it  results  in  a  better  match  to  the  theoretical  expert’s  model ’  (1991,  p.  iii,  emphasis 
added).  However,  the  participants  in  these  studies  were  generally  not  experts  in  operations.  Thus,  they  probably 
had  no  competing  model  as  expert,  trained  process  control  operators  might  have.  Training  was  noticeably  weak  in 
these  studies.  Yet  training  is  the  primary  source  of  the  mental  model  developed  by  professional  process  control 
operators. 

Some  of  these  issues,  notably  the  use  of  EID  displays  by  professional  operators,  were  addressed  in  the  research  on 
the  Rankine  cycle  display,  which  is  discussed  in  the  next  section. 

S.3.2.2  Rankine  Cycle  EID  Display 

Introduction 


Abstract  function  representations  of  any  process  plant  (such  as  a  NPP)  can  be  built  on  the  conservation  of  energy 
and  mass  and  on  thermodynamic  principles.  One  example  is  the  Rankine  cycle  display  (Beltracchi,  1987,  1989, 
1995).  Numerous  other  examples  of  such  displays  were  developed  (Duncan  and  Praetorius,  1992;  Hollnagel  et  al., 
1984),  which  show  important  aspects  of  the  process  cycle  (e.g.,  heat  source  and  heat  sink)  integrated  into  one 
display.  The  Rankine  cycle  display  was  incorporated  into  the  information  system  of  the  Experimental  Breeder 
Reactor  (EBR  II)  at  Argonne  National  Laboratory  (Lindsay,  1990).  The  Rankine  cycle  display,  its  thermodynamic 
rationale,  and  its  relationship  to  EID  was  described  by  Vicente  et  al.  (1996). 

The  general  functioning  of  a  typical  NPP,  the  thermodynamics  behind  the  Rankine  display,  the  ideal  Rankine 
cycle,  and  the  Rankine  cycle  EID  display  are  discussed  below. 
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Figure  5.7  Boiling  Water  Reactor  (BWR) 


Discussion  of  NPP  in  Physical  Terms 

Figure  5.7  shows  a  top-level  view  of  a  boiling  water  reactor  (BWR).  The  heat  source  is  the  nuclear  reactor,  in 
which  water  is  boiled  directly  in  the  reactor  vessel,  creating  steam.  In  a  BWR,  a  recirculation  loop  and  pump 
provides  forced  circulation  and  allows  higher  power  levels.  Steam  leaves  the  reactor  and  flows  down  the  main 
steam  lines  to  the  steam  turbine,  which  is  commonly  used  in  both  fossil  fuel  and  NPPs  to  convert  the  thermal 
energy  of  the  steam  to  rotational  energy.  An  electrical  generator  is  attached  to  the  turbine  shaft  and  converts  the 
rotational  energy  to  electrical  power.  After  the  steam  passes  through  the  turbine,  it  is  cooled  and  condensed  into 
water  in  the  main  condenser.  The  condenser  is  usually  supplied  with  river,  lake,  or  ocean  water  for  cooling.  After 
cooling,  the  water  is  pumped  back  to  the  reactor  to  complete  the  cycle.  As  will  be  discussed  later,  this  process  is  a 
thermodynamic  Rankine  cycle  (named  after  William  J.  M.  Rankine,  a  Scottish  engineer). 

Figure  5.8  presents  a  top-level  version  of  a  pressurized  water  reactor  (PWR).  In  this  type  of  reactor,  the  primary 
water,  which  is  heated  in  the  reactor  vessel,  is  kept  under  high  pressure  (typically  about  2250  psia)  by  a  pressurizer 
to  prevent  boiling  and  is  circulated  through  a  steam  generator  by  reactor  coolant  pumps.  The  hot  primary  water  (at 
about  600°F)  serves  as  the  heat  source  to  the  steam  generator,  where  secondary  water  is  boiled.  The  primary  and 
secondary  water  do  not  mix.  The  steam  produced  in  the  steam  generator  then  flows  down  the  main  steam  lines  to 
the  turbine-generator,  and  condenser.  After  cooling,  the  water  is  pumped  back  to  the  steam  generator  to  complete 
the  cycle. 

Discussion  of  Water 


Water  is  used  in  both  BWRs  and  PWRs  as  the  reactor  coolant  and  the  fluid  that  is  boiled  and  sent  to  the  turbines  to 
produce  electric  power.  The  water  exists  as  a  liquid,  a  gas  (steam),  and  a  combination  of  the  two.  Its  particular 
state  at  various  locations  of  the  process  systems  is  important  for  both  production  and  safety.  From  a 
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Figure  5.8  Pressurized  Water  Reactor  (PWR) 


thermodynamic  standpoint,  the  state  of  water  can  be  described  using  temperature,  pressure,  and  entropy,  as  shown 
in  Figure  5.9.  Temperature  (T)  is  on  the  y-axis  and  entropy  (s)  is  on  the  x-axis.  Entropy  is  a  mathematically 
defined  property  of  a  substance  that  increases  as  heat  is  added.  It  is  a  measure  of  the  disorder  of  a  system  at  the 
microscopic  level  (i.e.,  molecules  vibrating  more  as  temperature  is  raised  or  as  water  boils).  Constant  pressure 
lines  (P,  to  P3)  also  are  shown. 

To  the  left  of  the  curve  in  Figure  5.9,  water  is  a  subcooled  liquid,  with  a  temperature  less  than  the  boiling  point  for 
both  unpressurized  water  and  the  primary  water  in  a  PWR.  To  the  right  of  the  curve,  the  water  is  superheated 
steam,  with  a  temperature  above  the  boiling  point.  Starting  at  the  curve  on  the  left  side,  water  is  saturated  with  a 
temperature  at  the  boiling  point  for  a  given  pressure.  As  pressure  increases  from  P,,  to  P3,  along  the  saturated 
water  line,  the  boiling  point  temperature  increases.  For  a  given  constant  pressure,  if  saturated  water  is  heated  (as 
in  a  BWR  reactor  vessel  or  PWR  steam  generator),  the  water  will  begin  to  boil  and  its  properties  will  move  along 
the  horizontal  constant  pressure  lines  on  Figure  5.9.  This  shows  the  quality  of  the  steam  going  from  0%  (saturated 
water)  through  10%  steam,  90%  steam,  to  100%  steam  on  the  saturated  steam  line  on  the  right  of  the  curve.  If 
more  heat  is  added,  the  steam  then  becomes  superheated. 

Ideal  Rankine  Cycle 

The  power  plant  steam  cycle  is  termed  a  Rankine  cycle  and  can  be  displayed  graphically  on  the  temperature  (T) 
versus  entropy  (s)  plot  for  water.  This  is  shown  in  Figure  5.10. 
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Figure  5.9  Thermodynamic  Diagram  of  Water  and  Steam 


Figure  5.10  shows  an  ideal  Rankine  cycle  with  no  thermodynamic  losses  (such  as  friction).  The  cycle  physically 
consists  of  a  boiler  of  some  type,  a  turbine  where  work  is  extracted  from  the  steam  (for  conversion  to  electricity),  a 
condenser  where  the  exhaust  steam  from  the  turbine  is  condensed  into  liquid,  and  a  pump  where  the  condensed 
water  is  pumped  back  to  the  boiler  to  begin  the  cycle  again.  Each  of  these  steps  is  shown  on  the  figure.  Step  b-c  is 
the  boiler  where  the  water  is  heated  along  a  constant  pressure  line.  The  water  is  first  heated  to  saturation,  then 
boiled  (following  the  constant  temperature  horizontal  line),  and  finally  the  steam  is  superheated,  following  the 
same  constant  pressure  line.  U.S.  reactors  do  not  superheat  their  steam,  but  rather  send  saturated  steam  (at  about 
99.25%  quality)  to  the  turbine.  The  area  under  the  line  b-c  represents  the  total  heat  or  energy  added  from  the 
boiler.  Step  c-d  represents  an  ideal  turbine  where  the  energy  is  extracted  from  the  steam  at  a  constant  value  of 
entropy.  A  real  turbine  contains  some  losses  (not  all  of  the  steam  energy  is  converted  to  useful  work),  and  hence, 
line  c-d  for  a  real  turbine  would  slope  to  the  right  as  entropy  increases.  Step  d-a  represents  the  condensation  of  the 
steam  to  water  in  the  condenser  across  the  constant  temperature  line,  and  then  some  subcooling  of  the  water  along 
a  constant  pressure  line  that  is  very  close  to  the  saturated  water  line.  The  area  under  line  d-a  represents  the  heat  or 
energy  rejected  to  the  heat  sink  in  the  condenser,  which  is  called  unavailable  heat  or  energy.  The  area  between  the 
lines  b-c  and  d-a  represents  the  energy  converted  to  work  in  the  turbine.  By  ratios  of  these  areas,  the  theoretical 
efficiency  of  the  process  can  be  calculated.  Step  a-b  represents  the  increase  in  pressure  of  the  water  in  the 
condensate  and  feedwater  pumps,  as  it  is  sent  back  to  the  boiler.  In  the  real  case,  there  would  be  some  losses  in  the 
pumps  and  some  increase  in  entropy,  causing  the  line  to  curve  slightly  to  the  right.  Processes  to  the  left  of  the 
curve  stay  very  close  to  the  saturated  liquid  line  in  this  figure,  due  to  the  relative  incompressibility  of  water. 

Discussion  of  Rankine  Cycle  Display 

Figure  5.1 1  shows  the  display  that  was  evaluated  in  the  experiment  described  in  Vicente,  Moray,  et  al.  (1996).  The 
display,  based  on  the  Rankine  cycle,. represents  a  PWR  plant  which  uses  the  following:  a  steam  generator,  a  two- 
stage  turbine  with  a  high-pressure  and  a  low-pressure  stage,  steam  reheated  and  then  superheated  in  between  the 
two  turbine  stages,  a  condensate  pump  and  a  main  feedwater  pump  to  return  the  condensed  water  to  the  steam 
generator,  and  two  stages  of  feedwater  heaters  to  preheat  the  feedwater  before  its  return  to  the  steam  generator. 
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Figure  5.10  Rankine  Cycle 


This  display  reflects  a  real  (rather  than  ideal)  Rankine  cycle,  with  additional  information  and  graphics 
superimposed  upon  it.  The  correspondence  of  the  display  shown  in  Figure  5.1 1  to  the  ideal  Rankine  cycle 
(Figure  5.10)  will  be  described  first  and  then  the  added  information  on  the  display  will  be  addressed.  For 
discussion  purposes,  the  letters  a  through  g  were  added. 

The  process  step  b-c  in  Figure  5.11  represents  the  addition  of  heat  and  boiling  in  the  steam  generator  similar  to  the 
ideal  step  b-c  in  Figure  5.10.  This  includes  heating  of  the  water  to  saturation  and  then  boiling  all  the  water  to 
100%  steam.  There  is  no  superheating  of  the  steam  in  the  steam  generator.  The  lines  to  the  left  of  the  saturated 
water  line  appear  different;  this  will  be  discussed  below.  Step  c-d  in  Figure  5.11  represents  extracting  energy  from 
steam  in  the  turbine.  After  the  first  drop  in  temperature  in  the  high  pressure  turbine,  reheat  is  added  to  the  steam 
to  first  bring  it  to  the  saturated  steam  line,  and  then  to  superheat  it.  Next,  temperature  drops  again  in  the  low 
pressure  turbine  down  to  point  d.  Step  d-a  represents  the  condensation  of  the  steam  to  water  in  the  condenser 
across  a  constant  temperature  line  and  then  some  subcooling  of  the  water.  Finally,  Step  a-b  represents  the  increase 
in  pressure  of  the  water  in  the  condensate  and  feedwater  pumps,  as  it  is  sent  back  to  the  steam  generator,  as  well  as 
the  feedwater  preheating  that  will  be  discussed  below. 

Figure  5.1 1  differs  noticeably  from  the  ideal  Rankine  cycle  in  one  respect.  The  two  lines  representing  extraction  of 
energy  from  steam  in  the  turbine  (along  step  c-d)  slant  to  the  right  rather  than  being  a  vertical  constant  entropy 
line,  due  to  various  losses  in  the  process. 

A  substantial  amount  of  additional  information  was  added  to  the  display  in  Figure  5.1 1 .  Points  and  information  on 
the  display  are  determined  by  measured  parameters  sensed  at  key  points  in  the  plant,  such  as  temperature,  pressure, 
and  water  levels.  As  parameter  values  change,  the  polygon  display  adjusts  to  these  points  as  appropriate. 
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Figure  5.1 1  Rankine  Cycle  Display 

(Source:  Reprinted  with  minor  modification  with  permission  from  Vicente,  K.,  Moray,  N.,  Lee,  J., 

Rasmussen,  J.,  Jones,  B.,  Brock,  R.,  and  Djemil,  T.,  ‘‘Evaluation  of  a  Rankine  Cycle  Display  for 
Nuclear  Power  Plant  Monitoring  and  Diagnosis,”  Human  Factors ,  38(3),  1996.  Copyright  1996  by 
the  Human  Factors  and  Ergonomics  Society.  All  rights  reserved.) 

While  the  display  is  based  on  a  standard  temperature-entropy  (T-s)  chart,  it  does  not  directly  display  the  entropy 
axis.  The  constant  pressure  lines  are  only  displayed  in  the  two-phase  central  portion  and  the  superheat  portion  of 
the  display,  since  they  would  all  be  too  crowded  together  in  the  subcooled  region.  Also,  the  portion  of  the  figure  to 
the  left  of  the  liquid  saturation  line  (letters  a,  b,  e,  and  f)  was  changed  to  represent  the  degree  of  subcooling  by 
distance  to  the  left  of  the  curve.  Thus,  points  a  and  b  in  Figure  5.1 1  appear  differently  from  Figure  5.10. 
Furthermore,  the  display  in  this  area  after  the  condenser  represents  the  following:  first,  a  pressure  increase  in  the 
condensate  pump,  next  an  increase  in  temperature  in  a  feedwater  heater,  then  an  increase  in  pressure  in  the 
feedwater  pump,  and  a  further  increase  in  temperature  in  a  second  feedwater  heater.  This  portion  of  the  display  is 
not  discussed  by  Vicente  et  al.  (1996). 

The  display  also  includes  information  on  the  primary  loop,  the  pressurizer,  some  liquid  levels,  and  relief  valves  on 
the  primary  and  secondary  sides,  as  follows.  The  temperatures  of  the  primary  loop  (reactor  coolant  system  or  RCS) 
and  the  heat  transfer  from  the  primary  to  secondary  sides  are  displayed  by  the  small  trapezoidal-shaped  figure  with 
the  line  e-f.  The  horizontal  line  from  point  e  represents  the  cold  leg  of  the  RCS.  It  is  drawn  at  the  temperature  of 
the  cold  leg,  Tc.  Also,  the  distance  from  the  saturation  curve  line  to  point  e  represents  the  amount  of  subcooling  in 
the  cold  leg.  This  is  an  important  feature  because  the  RCS  in  a  PWR  should  always  remain  subcooled.  Similarly, 
the  line  from  point  f  represents  the  hot  leg,  and  is  drawn  at  the  hot  leg  temperature,  TH.  The  subcooling  is  less  for 
the  hot  leg  than  the  cold  leg.  The  distance  between  TH  and  Tc  is  proportional  to  the  heat  transferred  from  the  RCS 
to  the  steam  generator.  From  this  trapezoid,  the  operator  can  infer  that  the  primary  system  is  properly  subcooled 
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and  is  correctly  transferring  heat  to  the  secondary  side.  Actual  values  are  not  included  here,  and  other  displays  or 
indicators  would  be  needed  to  show,  for  example,  the  subcooling  margin  (how  far  away  the  primary  is  from 
saturation).  As  an  example,  a  typical  SPDS  alarm  for  low  subcooling  margin  is  set  at  32°F  to  allow  a  margin  for 
instrument  uncertainty. 

The  horizontal  lines  within  the  saturation  curve  represent  the  steam  generator  (the  line  left  from  point  c),  the 
condenser  (the  line  left  from  point  d),  and  the  pressurizer  (the  line  left  from  point  g).  These  lines  are  at  the  correct 
temperature  and  pressure  for  each  of  those  components.  For  example,  the  lme  left  from  point  g  for  the  pressurizer 
is  at  a  pressure  of  about  2250  psia  and  a  temperature  of  about  653  °F.  The  water  level  in  each  of  these  three 
components  is  represented  by  the  small  square  box  on  the  horizontal  line.  The  state  of  “all  water”  corresponds  to 
the  box  at  the  left  on  the  saturated  water  curve  and  “all  steam”  (no  water)  is  with  the  box  at  the  right  on  the 
saturated  steam  curve.  In  between,  the  position  of  the  box  represents  a  percent  level. 

The  P&ID-like  graphic  under  the  Rankine  curves  represents  the  circulating  water  system  that  cools  the  condenser. 
Also,  relief  valve  icons  were  added  for  a  pressurizer  relief  valve  and  a  steam  generator  relief  valve,  and  a  main 
steam  line  valve  icon  was  added  on  the  horizontal  line  for  steam  generator.  These  changed  color  when  open  or 
closed.  Various  other  portions  and  lines  on  the  display  were  color  coded  to  aid  the  operators. 

Thus,  while  the  display  of  Figure  5.1 1  is  based  primarily  on  the  Rankine  cycle,  significant  additions  and  changes 
were  made  to  give  more  information  relevant  to  various  aspects  of  the  NPP. 

Evaluation  of  EID-Type  Displays 

According  to  Lindsay  (1990),  the  use  of  this  EID  type  of  display  has  certain  advantages.  The  display  takes 
advantage  of  human  pattern  recognition.  However,  unlike  simpler  configural  displays,  the  underlying  pattern  is 
related  to  plant  thermodynamics.  The  display  also  integrates  important  information  together  in  one  display,  which 
gives  a  significant  overview  of  plant’s  performance.  A  properly  trained  operator  can  use  pattern  recognition  to 
detect  plant  disturbances  by  distortions  in  the  pattern. 

The  effects  of  the  Rankine  cycle  display  on  diagnosing  events  have  been  investigated  (Moray  et  al.,  1993;  Moray  et 
al.,  1994;  and  Vicente  et  al.,  1996).  The  Rankine  display  was  compared  with  single-parameter,  linear-meter 
displays  and  a  display  that  included  both  single-parameter  meter  display  (Figure  5.12)  and  a  pressure  vs. 
temperature  plot.  The  study  used  three  groups  of  42  participants  each:  undergraduate  engineering  students, 
graduate  engineering  students,  and  professional  NPP  operators. 

The  procedure  required  the  participants  to  monitor  nine  transients  for  two  to  four  minutes  on  a  desktop  VDU.  The 
data  for  the  transients  were  obtained  from  a  PWR  training  simulator.  Each  participant  used  only  one  type  of 
display.  Following  each  transient,  the  participants  were  given  either  a  quantitative  recall  task  (e.g.,  “what  is  the 
value  of  parameter  x”),  or  a  qualitative  recall  task  (e.g.,  “did  the  primary  coolant  remain  sub-cooled  for  the 
duration  of  the  trial?”),  and  asked  to  indicate  whether  something  was  abnormal.  If  something  abnormal  did  occur, 
they  were  asked  to  make  a  diagnosis. 

For  accuracy  of  detection,  a  significantly  greater  percent  correct  was  obtained  with  the  Rankine  display  in 
comparison  to  the  other  two.  In  addition,  the  NPP  operators  were  better  at  detection  than  the  other  two  groups. 

The  Rankine  display  also  led  to  more  accurate  diagnoses  than  the  others,  and  operators  were  more  accurate  than 
the  other  groups. 

The  Rankine  display  also  led  to  better  qualitative  recall,  but  the  difference  was  not  significant.  Quantitative  recall 
was  better  with  the  meter  display.  The  authors  concluded  that  diagnosis  is  more  associated  with  qualitative 
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Figure  5.12  Simulated  Analog  Meter  Display 

(Source:  Reprinted,  with  minor  modification,  with  permission  from  Vicente,  K.,  Moray,  N  ,  Lee,  J., 
Rasmussen,  J.,  Jones,  B.,  Brock,  R.,  and  Djemil,  T.,  “Evaluation  of  a  Rankine  Cycle  Display  for 
Nuclear  Power  Plant  Monitoring  and  Diagnosis,”  Human  Factors,  38(3),  1996.  Copyright  1996  by  the 
Human  Factors  and  Ergonomics  Society.  All  rights  reserved.) 


information  rather  than  quantitative  information.  The  Rankine  display  was  better  at  supporting  the  diagnostic 
performance  of  experts. 

It  is  interesting  that,  despite  the  performance  effects  noted  above,  the  operators  did  not  like  the  Rankine  display 
because  they  were  unfamiliar  with  it.  They  also  claimed  that  they  missed  the  alarm  systems  that  they  normally  use 
for  diagnosis.  In  their  actual  plants,  the  alarms  are  the  first  indication  of  a  disturbance,  and  their  pattern  typically 
provides  information  about  the  nature  of  a  disturbance. 

The  authors  suggested  that  failures  in  instrumentation  can  affect  performance  and  interpretation  of  the  display. 
Figure  5.13  shows  two  displays:  one  representing  a  real  LOCA  and  the  other  a  failed  instrument. 

While  only  one  of  thirty-five  instrumentation  points  failed,  the  display  was  very  difficult  to  interpret.  Moray  et  al. 
(1994)  noted  that 


The  calculations  involved  in  coupling  information  from  many  sources  to  produce  the  DPIs  [direct  perception 
interfaces]  are  extremely  vulnerable  to  certain  classes  of  failures.  Little  or  no  research  exists  in  this  aspect  of  DPIs, 
and  is  urgently  needed.  The  real  advantages  of  such  displays  during  normal  operation  and  during  many  classes  of 
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(a)  Rankine  cycle  display  for  loss  of  coolant  accident 


Figure  5.13  Rankine  Cycle  Display  of  Abnormalities 

(Source:  Reprinted  with  permission  from  Moray,  N.,  Lee,  J.,  Vicente,  K.,  Jones,  B.,  and  Rasmussen,  J.,  “A 
Direct  Perception  Interface  for  Nuclear  Power  Plants,”  Proceedings  of  the  Human  Factors  and  Ergonomics 
Society  38th  Annual  Meeting ,  1994.  Copyright  1994  by  the  Human  Factors  and  Ergonomics  Society.  All 
rights  reserved.) 


transients  may  be  more  than  offset  if  they  collapse  under  other  classes  of  abnormalities.  An  understanding  of  the 
failure  modes  of  DPIs  (as  distinct  from  the  failure  modes  of  the  plant  itself)  is  as  necessary  as  an  understanding  of 
their  design  for  normal  conditions  Existence  proofs  of  interface  designs  arc  no  substitute  for  full  empirical 
evaluations,  and  yet  very  few  advanced  DPIs  have  been  exhaustively  evaluated  over  a  wide  range  of  abnormal 
conditions.  It  is  clear  from  our  results  that  DPIs  and/or  ecological  interfaces  can  fail  in  catastrophic  ways,  and  it  is 
probable  that  such  failures  may  be  particularly  dangerous  in  large  richly  coupled  systems,  (p.  485) 
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To  briefly  summarize  the  literature  for  the  Rankine  cycle  display,  it  clearly  reflects  the  principles  of  EID  and  has 
been  used  in  designs  of  new  power  plants  (see  also  the  development  of  Toshiba  HSI,  below)  and  by  professional 
operators  (see  the  EBR II  application).  There  is  currently  only  one  empirical  evaluation  of  the  display,  which 
despite  some  negative  operator  comments,  was  associated  with  better  performance  of  detection  and  diagnosis  than 
other  types. 

Despite  the  success  of  the  experiment  in  beginning  to  address  a  real  research  need,  there  may  be  several  limitations 
to  the  study.  The  first  issue  is  generalization  of  the  findings  as  representative  of  the  strengths  and  weaknesses  of 
the  design  approaches  they  intended  to  represent.  There  are  two  problems  with  the  study.  First,  if  the  research  is 
to  provide  a  valid  comparison  of  an  EID  approach  and  an  alternative,  the  alternative  approach  to  identifying 
information  requirements  was  not  clearly  defined,  and  the  alternative  displays  were  not  well  designed.  Second, 
there  is  a  confound  between  information  presented  and  display  representation  or  format.  That  is,  any  difference 
between  the  alternative  experimental  conditions  may  be  due  to  the  information  requirements  analysis  (EID  vs. 
other)  or  to  the  different  format  (graphical  display  vs.  single  meter  displays).  Contrasting  the  Rankine-cycle 
display  with  a  set  of  35  individual  analog  meter  displays  (shown  in  Figure  5.12)  is  a  biased  comparison  -  even 
when  augmented  with  a  pressure-temperature  plot. 

The  second  issue  is  identification  of  information  requirements.  What  cells  of  the  abstraction  hierarchy  does  the 
display  represent?  What  information  needs  is  this  display  meeting?  What  analysis,  ecological  or  otherwise,  led  to 
the  selection  of  the  Rankine  cycle  as  an  appropriate  mental  model  for  operators  to  conduct  their  tasks?  Are 
operators  sufficiently  familiar  with  thinking  of  the  plant  on  these  terms  to  effectively  use  the  display?  Another  way 
to  approach  the  question  of  information  requirements  is  noted  below  in  item  (1)  under  General  Issues  to  Consider 
Relevant  to  EIDs. 

The  third  issue  is  generalization  of  the  findings  to  other  tasks,  both  complex  and  routine.  The  study  was  not  a  fully 
interactive  dynamic  simulation  and  operators  did  not  have  the  full  HSI  available  to  them.  Thus,  potential 
interactive  effects  with  the  Rankine  display  in  the  context  of  other  plant  HSIs  could  not  be  evaluated;  nor  could  its 
effects  on  other  operational  tasks. 

A  fourth  issue  relates  to  proper  interpretation  the  experiments,  results.  It  was  noted  that  the  operators  commented 
that  they  did  not  like  the  Rankine  EID  display  and  would  have  preferred  to  have  some  of  their  normal  HSI 
available,  namely  the  alarm  system.  Perhaps  they  would  have  preferred  to  have  their  entire  display  system  rather 
than  the  EID  display,  and  would  have  done  better  with  it.  In  fact,  given  the  low  absolute  performance  values  (78% 
for  detection  and  45%  for  diagnosis)  recorded  by  the  experienced  operators,  they  may  have  worked  better  than  this 
with  their  typical  HSI  suite  rather  than  with  the  Rankine  display. 

General  Issues  to  Consider  Relevant  to  EIDs 

This  experiment,  and  particularly  the  Rankine  display,  show  a  few  important  points  to  consider  that  are  relevant  to 
the  design  and  implementation  of  EID  displays.  Where  appropriate,  the  more  generic  aspects  of  these  issues  were 
included  in  Section  5.6,  General  Summary  and  Issues. 

(1)  In  developing  a  higher-level  display,  one  needs  a  clear  definition  of  what  higher-level  functions  and  goals  are 
to  be  addressed.  As  an  example,  are  you  addressing  the  goal  of  safety  or  electric  production,  or  both?  For  the 
goal  of  safety,  the  NRC  established  five  critical  safety  functions  needed  to  achieve  overall  safety  in  NUREG- 
1342  (NRC,  1988),  namely,  reactivity  control,  reactor  core  cooling  and  heat  removal,  RCS  integrity, 
radioactivity  control,  and  containment  conditions.  Tables  2  and  3  of  the  NUREG  give  sample  parameters  to  be 
monitored  to  address  these  functions  for  both  BWRs  and  PWRs. 
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The  experiment  did  not  clearly  define  which  goals  or  functions  the  EID  was  to  address.  Examining  the  five 
critical  safety  functions,  one  sees  that  three  (reactivity  control,  radioactivity  control,  and  containment 
conditions)  are  not  addressed  at  all  in  the  display.  RCS  integrity  and  reactor  core  cooling  and  heat  removal 
are  addressed  somewhat,  but  all  of  the  recommended  parameters  are  not  included.  For  example,  reactor  vessel 
level,  core  exit  thermocouples,  and  residual  heat  removal  (RHR)  flow  are  not  included.  Also,  the 
discrimination  of  some  of  the  provided  important  parameters  (reactor  pressure,  subcooling  margin,  and 
pressurizer  level)  is  not  fine  enough  to  detect  relatively  small  changes,  and  there  does  not  appear  to  be  any 
trending  capability. 

Thus,  while  the  Rankine  display  gives  some  higher  level  functional  information,  it  does  not  appear  to  provide 
a  sufficient  amount,  and  not  nearly  as  much  as  a  typical  NPP  SPDS  implementation.  Perhaps  a  useful 
experiment  would  compare  a  typical  SPDS  with  an  improved  EID  display  based  on  the  Rankine  cycle.  The 
improvements  would  be  to  incorporate  the  five  SPDS  functions  and  the  related  important  parameters.  This 
certainly  appears  to  be  a  feasible  experiment. 

(2)  For  any  higher-level  display,  where  the  information  is  sophisticated  and  a  large  amount  of  it  is  consolidated 
into  one  or  a  few  graphics,  training  becomes  very  important.  One  needs  to  understand  all  of  the  aspects  of  the 
display  and  how  it  reacts  to  various  operational  transients,  accidents,  and  instrument  failures.  Further,  many 
of  the  items  in  the  display  are  not  labeled  (e.g.,  TH  and  Tc)  and  would  be  burdensome  to  estimate  unless 
operators  were  highly  familiar  with  the  display. 

(3)  The  effect  of  failures  of  instrumentation  on  EID  displays  was  recognized  as  a  potentially  significant  problem 
that  needs  to  be  addressed.  There  are  several  subissues  associated  with  this  problem. 

•  Can  operators  detect  a  failure  of  instrumentation? 

•  Can  failures  of  instruments  result  in  representations  that  are  interpreted  by  operators  as  real  process 
failures;  and,  perhaps  more  importantly,  can  real  process  failures  be  misinterpreted  as  failures  of 
instruments? 

•  If  operators  detect  a  failure,  should  use  of  the  display  be  suspended? 

•  Since  the  display  integrates  many  parameters  into  a  single  display,  what  is  the  effect  on  operations  of  its 
loss,  and  how  effectively  can  operators  switch  to  backup  displays? 

(4)  The  integration  of  a  new  and  significantly  different  EID  into  the  remainder  of  the  standard  HSI  of  a  CR  is  an 
important  consideration  that  needs  attention. 

(5)  The  issue  of  the  operator’s  acceptance  of  a  new  and  different  type  of  display  (such  as  an  EID)  also  is 
important,  as  shown  by  the  operator’s  comments  during  the  experiment.  One  interesting  idea  was  introducing 
the  displays  initially  in  training  and  then  perhaps  gradually  introducing  them  into  the  plant. 

S.3.2.3  Toshiba  Advanced  BWR  HSI 

The  EID  approach  to  interface  design  is  guiding  Toshiba’s  development  of  an  intelligent  interface  for  an  advanced 
BWR  information  system.  A  series  of  papers  (discussed  below)  described  the  development  of  the  information 
system,  giving  examples  of  the  displays,  and  discussing  some  validation  experiments.  This  application  of  EID  is 
interesting  because  it  represents  an  effort  to  develop  an  entire  NPP  information  system  based  on  EID  principles. 
However,  while  the  importance  of  the  evaluation  of  EID  displays  was  identified,  little  information  on  the 
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Table  5.  3  Operator  Cognitive  Activity 


Level  of 
Abstraction 

Plant 

Level  of  Aj 

System 

jgregation 

Equipment 

Component 

Functional  Purpose 

X 

X 

Abstract  Function 

X 

X 

X 

Generalized  Function 

X 

X 

Physical  Function 

X 

X 

effectiveness  of  the  interface  is  available. 

The  basis  for  the  design  is  a  cognitive  task  analysis  reflecting  the  abstraction  hierarchy  (Itoh  et  al.,  1990).  As  part 
of  this  effort,  operators  engaged  in  situation  assessment,  diagnosis,  and  response  planning  for  simulated  plant 
disturbances  involving  multifault  scenarios.  The  scenarios  were  designed  to  elicit  knowledge-based  behavior. 
Operators  were  asked  to  verbalize  their  thought  processes.  The  data  were  analyzed  and  sorted  into  the  cells  in  the 
A-A  matrix.  Table  5.3  shows  the  envelope  of  cognitive  activity.  The  correlation  between  the  higher  levels  of 
abstraction  and  higher  levels  of  aggregation  is  noteworthy  (e.g.,  operators  did  not  reason  at  the  plant  level  in  terms 
of  physical  form,  nor  did  they  reason  about  components  at  the  functional  level). 

The  authors  concluded  that  information  reflecting  these  cells  is  needed,  at  a  minimum,  in  the  interface.  At  each 
level  of  abstraction,  the  following  types  of  information  were  preliminarily  identified  as  necessary: 

•  Functional  purpose  -  the  goal  of  each  function  should  be  expressed  in  flow  of  mass,  energy,  and  information. 
No  information  on  equipment  or  components  is  necessary. 

•  Abstract  function  -  information  should  be  presented  on  the  flow  and  distribution  of  mass  and  energy  (the 
relationship  between  origins  and  destinations  of  substance  or  energy  flows). 

•  Generalized  function  -  information  should  be  presented  on  the  relationships  between  objects,  input-output 
relationships,  and  information  on  the  operating  sequence. 

•  Physical  function  -  information  should  be  centered  around  the  properties  of  objects  and  can  be  categorized  in 
terms  of  underlying  physical  processes. 

•  Physical  form  -  information  should  be  provided  on  component  status  and  appearance,  and  how  it  is  accessed. 
For  specific  purposes,  operators  may  need  information  on  component  settings  and  parts  drawings. 

Examples  of  displays  supporting  some  of  these  levels  were  presented  elsewhere  (Sakuma  et  al.,  1990;  Monta  et  al., 
1991).  Monta  et  al.  (1991)  also  discuss  an  intelligent  “advisor”  that  can  generate  strategies  and  procedures  after 
faults. 
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(Source:  Reprinted  with  permission  from  Monta,  K.,  Takizawa,  Y.,  Hottori,  Y.,  Hayashi,  T.,  Sato,  N.,  Itoh, 
J,,  Sakuma,  A.  and  Yoshikawa,  E.  (1991),  “An  Intelligent  Man-Machine  System  for  BWR  Nuclear  Power 
Plants,”  Proceedings  of  the  AI9I  Frontiers  in  Innovative  Computing  for  the  Nuclear  Industry.  LaGrange 
Park,  IL:  American  Nuclear  Society,  pp.  383-392.  Copyright  1991  by  the  Valley  Litho,  Idaho  Falls,  Idaho. 
All  rights  reserved.) 


Figure  5.14  is  an  example  of  a  production  goal  display  from  Monta  et  al.  (1991).  The  uppermost  section  of  the 
display  represents  the  flow  of  electrical  production.  Its  center  shows  the  process  for  converting  the  energy  and 
includes  a  display  based  on  the  Rankine  cycle.  The  lower  part  of  the  display  depicts  the  mass  balance  for  the 
reactor  vessel  and  the  condenser,  showing  inlet  and  outlet  flows  and  water  levels. 

This  display  was  intended  to  be  an  externalized  mental  model  of  the  process.  Thus,  workload  should  be  reduced 
because  operators  do  not  have  to  recall  these  relationships  from  memory  and  mentally  compute  information  related 
to  higher-level  goals. 

The  display  was  tested  using  a  full-scope,  advanced  boiling  water  reactor  (ABWR)  simulator  (Monta  et  al.,  1992). 
Six  experienced  crews  of  utility  operators  were  presented  with  simulated  plant  disturbances.  Subjective  evaluations 
of  the  interface  were  obtained  following  each  trial.  The  operators’  evaluations  were  generally  positive;  however, 
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Figure  5.15  Toshiba  EID  Display 

(Source:  Reprinted  from  Itoh,  J.,  Sakuma,  A.  and  Monta,  K.,  “An  Ecological  Interface  for 
Supervisory  Control  of  BWR  Nuclear  Power  Plants,”  Control  Engineering  Practice,  3(2),  1995, 
Copyright  1995,  with  permission  from  Elsevier  Science.) 


more  detailed  results  were  not  given.  Itoh  et  al.  (1995)  gave  additional  information  on  the  evaluation  of  the 
Toshiba  EID  display  (Figure  5.15).  This  EID  was  subjected  to  an  empirical  evaluation  (Takizawa  et  al.,  1994). 

Itoh  et  al.  (1995)  presented  a  typical  comment  from  operators  participating  in  the  empirical  evaluation  of  the 
display.  “EID  display  would  suit  the  novice  operator  training  to  build  up  his/her  mental  model.  However,  it  is  too 
crowded  with  information  to  easily  understand  the  situation”  (p.  238).  This  is  not  a  very  comforting  result. 
Operators  may  have  experienced  difficulty  understanding  the  plant  situations  shown  in  the  displays.  An 
independent  examination  of  the  display  shows  that  many  aspects  are  not  self-evident  and  would  probably  require 
substantial  training  before  operators  could  use  it  in  operational  situations.  This  also  illustrates  the  importance  of 
evaluating  the  relationship  between  the  representation  in  the  display  and  the  operator’s  mental  model  of  the 
domain,  as  well  as  its  relationship  to  training. 

Itoh  et  al.  (1995)  also  discuss  the  difficulties  of  evaluating  this  type  of  interface.  The  authors  stated  that 
“evaluation  of  the  man-machine  interface,  especially  for  new  designs  such  as  EID,  is  one  of  the  most  important 
issues,  since  it  should  guarantee  that  the  design  objectives  have  been  completely  achieved  before  interface  usage. 
Its  importance  as  well  as  its  difficulty  increases  for  the  present  case,  because  the  main  objective  is  to  support  the 
operator’s  knowledge-based  behavior  to  cope  with  unanticipated  events”  (p.  237).  Methods  are  needed  to  evaluate 
the  technical  accuracy  as  well  as  the  human  factors  and  performance  aspects  of  the  design’s  use. 
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In  summary,  Toshiba  has  put  substantial  resources  into  developing  and  testing  an  EID-based  display  system  for  the 
ABWR  and  believes  that  it  will  provide  benefits  over  traditional  displays.  Since  the  ABWR  has  now  been 
operating  for  some  time,  operating  experience  on  the  EID  should  soon  become  available.  This  would  provide 
valuable  information  on  its  use  in  real-life  situations. 

5.3.2.4  Other  EID  Studies 

This  section  discusses  nine  additional  studies  of  EID-type  displays  that  addressed  a  variety  of  different 
applications. 

Edlund  and  Lewis  (1994)  compared  five  types  of  displays  for  controlling  a  simulation  of  a  simplified  steam  plant. 
Eighteen  participants  were  given  the  task  to  maintain  boiler  level  and  turbine  speed  in  response  to  variations  in 
load.  When  a  fault  occurred,  they  were  to  stop  the  simulation  and  make  a  diagnosis.  The  five  displays  (see  Figure 
5.16)  used  in  the  test  were 

•  Dials  display  -  Traditional  gages  were  used  to  present  process  parameters.  No  information  on  constraints  or 
relationships  between  parameters  was  given  (see  Fig.  5.16a). 

•  Fluid-tanks  display  -  This  display  represents  boiler  and  turbine  levels.  Feedwater,  fuel,  and  make-up  controls 
are  shown  as  valves,  which  control  flow  in  the  system.  Pipes  and  valves  were  animated  to  depict  flow 
characteristics.  Unlike  the  mimic  display,  the  effects  of  parameters  were  not  displayed  as  functions  of 
commanded  pumping  rates  (e.g.,  feedwater  pump  rate).  The  fluid-tank  display  is  closest  to  the  EID-type 
displays  discussed  above  (see  Fig.  5.16b). 

•  Seesaw  display  -  The  process  is  displayed  as  a  seesaw  with  balances  representing  the  fuel,  feedwater,  and 
makeup  pump  rates.  The  seesaw  display  is  interesting  because  it  incorrectly  represents  the  dynamics  of  the 
process  (see  Fig.  5.16c). 

•  Mimic  display  -  The  same  process  parameters  were  presented  as  in  the  dials  display,  but  the  mimic  provided  a 
context  revealing  the  relationships  between  them  and  their  constraints  (such  as  direction  of  flow).  Its 
disadvantage  was  that  the  operator’s  task  is  to  control  states  rather  than  individual  components.  This 
information  is  not  given  and  the  operator  must  mentally  determine  how  to  integrate  it  to  manage  the  states 
(see  Fig.  5.16d). 

•  Object  display  -  The  process  parameters  are  integrated  into  a  single-object  display  (see  Fig.  5.16e). 

The  general  results  were  that  the  best  performance  was  obtained  with  the  fluid-tank,  dial,  and  mimic  displays.  The 
integrated  object  display  fared  more  poorly.  The  authors  hypothesized  that  this  was  because  it  presents  state 
information  with  no  context  or  process  information.  The  seesaw  display  led  to  the  worst  performance,  which 
emphasizes  the  importance  of  accurately  mapping  the  features  of  the  display  to  the  dynamics  they  are  intended  to 
represent. 

In  another  study,  an  EID  display  was  compared  with  a  mimic  display  for  a  chip  refiner  in  a  pulp  factory  (Brehmer 
et  al.,  1995).  Fifteen  process  operators  used  the  displays  in  a  process  simulation.  The  operators  were  divided  into 
three  groups:  P&ID,  EID,  and  EID  with  special  training.  The  EID  display  was  associated  with  better  performance, 
and  interestingly,  no  special  training  was  found  to  be  necessary. 
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(c)  Seesaw  display 


(e)  Object  display 


Figure  5.16  Edlund  and  Lewis  Displays 

(Source:  Reprinted  with  permission  from  Edlund,  C.  and  Lewis,  M.,  “Comparing  Ecologically 
Constrained  and  Conventional  Displays  in  Control  of  a  Simple  Steam  Plant,”  Proceedings  of  the 
Human  Factors  and  Ergonomics  Society  38th  Annual  Meeting ,  1994.  Copyright  1994  by  the  Human 
Factors  and  Ergonomics  Society.  All  rights  reserved.) 
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Similarly,  Meshkati  et  al.  (1994)  evaluated  the  effectiveness  of  an  ecological  interface  for  handling  process  control 
disturbances.  They  found  that  the  interface  led  to  significantly  more  accurate  diagnosis  of  events  and  faster 
response  than  a  computer  emulation  of  the  current  traditional  console. 

Reising  and  Sanderson  (1996)  conducted  a  detailed  analysis  of  a  pasteurization  plant  using  the  abstraction 
hierarchy.  They  found  the  following  important  factors  for  EID: 

•  Limitations  in  sensor  technology  may  limit  the  designers’  ability  to  represent,  in  displays,  the  information 
found  in  the  analysis  of  the  abstraction  hierarchy. 

•  The  displays  of  higher  order  variables  and  their  relationships  are  based  on  computations  using  lower-level 
information,  such  as  volume  and  temperature.  The  information  displayed  will  be  limited  by  the  quality  of  the 
lower-level  information. 

•  Sensor  failures  or  errors  may  affect  the  form  and  content  of  the  display. 

One  important  contribution  of  a  thorough  analysis  of  abstraction  hierarchy  is  that  it  can  reveal  not  only  the 
operators’  information  needs,  but  what  sensors  are  needed  and  where  in  the  plant  they  should  be  placed.  In  fact, 
the  authors  state  that  EID  is  “in  a  position  to  bring  about  a  profoundly  user-centered  approach  to  system  design  in 
which  the  ultimate  information  needs  of  human  controllers  drive  the  engineering  agenda  of  sensors  and 
instrumentation  in  a  feed-forward  manner”  (p.  296). 

Based  on  their  experience  in  applying  the  process,  Reising  and  Sanderson  (1996)  concluded  that  building  an 
abstraction  hierarchy  is  an  iterative  process  that  required  expertise  in  thinking  in  terms  of  abstraction  hierarchies 
and  in  conducting  the  analyses.  The  knowledge  of  domain  experts  also  is  necessary.  In  addition,  keeping  track  of 
the  links  in  the  hierarchy  was  very  difficult  and  tools  to  support  such  analyses  are  needed.  Lack  of  such  support 
limits  their  use. 

Shively  (1995)  examined  the  use  of  ecological  approaches  to  91 1  dispatcher  interfaces.  Ten  experienced 
dispatchers  completed  eight  trials,  half  with  a  traditional  text  display  and  half  with  a  graphic  EID  display.  Their 
task  was  to  assign  the  most  appropriate  resource  to  each  incident.  Time  and  ratings  of  situation  awareness, 
workload,  and  preference  were  collected.  Response  times  did  not  differ;  however,  the  operators  were  very 
experienced  with  the  text  display  and  had  little  training  on  the  EID  display.  There  were  trends  toward  improved 
situation  awareness  and  reduced  workload  with  the  EID  display.  The  EID  display  was  preferred  as  well. 

In  the  medical  domain,  an  EID  display  for  hemodynamic  monitoring  was  compared  with  a  conventional  display 
(Effken  et  al.,  1992).  The  display  provides  clinical  patient  information,  including  current  state  and  goal  state. 

Data  to  determine  the  relevant  information  were  obtained  from  expert  clinicians.  From  this  information,  an 
integrated  object  display  was  developed.  The  display  represents  pressures  by  interconnected  balloons  for  cardiac, 
venous,  and  arterial  compartments,  and  cardiac  flow  as  a  bellows.  This  display  reveals  the  functional  relationship 
of  the  hemodynamic  system.  Traditionally,  this  information  is  monitored  using  strip  chart  displays  of  individual 
parameters. 

Thirteen  participants  at  three  experience  levels  used  the  displays  to  handle  simulated  clinical  events  using  a 
within-subjects  design.  Performance  measures  included  time  to  initiate  treatment,  time  to  achieve  target  range, 
percent  of  time  in  target  range,  and  the  number  of  drugs,  drug  changes,  and  combined  drug  and  dose  changes. 
Compared  to  the  strip  chart  display,  the  EID  display  performed  better,  showing  the  following:  higher  percent  of 
problems  corrected,  greater  percent  time  in  target  range,  fewer  drugs,  drug  changes,  and  combined  drug  and  dose 
changes.  There  was  an  interaction  with  time  to  initiate  treatment,  such  that  the  more  experienced  participants  took 
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performed  in  parallel  and  some  sequentially.  This  temporal  dimension  to  operations  may  be  lacking  in  the  A-A 
analysis,  except  insofar  as  time  is  reflected  in  higher-level  information,  such  as  rates. 

Additionally,  the  EID  approach  seems  to  be  built  around  functional  decomposition  of  means-ends  relationships. 
How  well  such  an  approach  will  support  common  tasks  or  the  handling  of  disturbances  that  are  highly 
proceduralized  needs  to  be  determined.  While  it  is  important  to  be  able  to  deal  with  unanticipated  and  unplanned 
events,  still  most  of  the  operator’s  activities  are  well  planned  and  the  information  system  needs  to  support  these 
activities  as  well. 

(3)  Information  Volume  and  Density 

Due  to  the  extensive  analytical  process  used  for  EIDs,  the  process,  including  the  A-A  matrix,  may  identify  more 
information  than  can  practically  be  displayed.  Also,  too  many  pages  may  be  required  to  satisfy  EID  information 
requirements  (see  also  discussion  of  density  in  Section  3.5).  Potentially  negative  impacts  of  designing  displays 
using  the  abstraction  hierarchy  are  the  potential  proliferation  in  the  number  of  pages  and  the  increase  in  the 
density  of  information  on  each  page. 

(4)  Integration  of  EID  Displays  into  the  Remainder  of  the  HSI 

Integrating  a  new  and  significantly  different  EID  display  into  the  remainder  of  the  standard  HSI  of  a  CR  is  an 
important  consideration  to  address. 

(5)  Critical  Testing  and  Evaluation  of  EID  Concepts 

There  has  not  been  much  thorough  research  that  carefully  assesses  the  various  aspects  of  EID,  e.g.,  information 
requirements,  effect  of  organization  of  information  along  functional  lines,  display  representation,  and  use  of 
analytical  redundancy.  Studies  tend  to  confound  these  characteristics  or  give  weak  assessments  of  the  contribution 
of  the  various  aspects  of  EID  concepts. 

Successfully  handling  unplanned  or  unanticipated  events  with  EIDs,  under  actual  operational  conditions  in 
complex  systems,  has  not  been  demonstrated. 

Further  research  also  is  necessary  to  more  clearly  identify  which  cells  of  the  A-A  matrix  are  important  to 
operations. 

(6)  Evaluation  of  Operating  Experience 

For  EID  displays  that  were  installed  in  operating  facilities,  there  needs  to  be  a  thorough  assessment  of  the 
operating  experience  and  how  it  applies  to  the  EID.  Actual  application  of  this  approach  to  NPP  HSI  design 
includes  the  Toshiba  ABWR  CR  (discussed  earlier),  a  NPP  feedwater  system  (Dinadis  and  Vicente,  1996),  and  the 
EBRII  (Lindsey,  1990). 

(7)  Internal  vs.  External  Mental  Models 

Carroll  and  Olson  (1988)  note,  “If  the  interface  suggests  or  reflects  an  appropriate  model,  then  the  user  could 
conceivably  learn  it  with  less  guidance  and  perform  with  fewer  errors.  The  question  is:  What  should  the  model 
be?”  (p.  56).  Compatibility  of  information  representation  and  operator  mental  models  is  characteristic  of  “user- 
centered  approaches  to  design  and  design  review.”  For  example,  the  high-level  design  review  principle  of  User 
Model  Compatibility  from  NUREG-0700,  Rev.  1  states  as  follows: 
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All  aspects  of  the  system  should  be  consistent  with  the  users’  mental  models  (understanding  and  expectations 
about  how  the  system  behaves  as  developed  through  training,  use  of  procedures,  and  experience).  All  aspects  of 
the  system  also  should  be  consistent  with  established  conventions  (i  e.,  expressed  in  customary,  commonplace, 
useful  and  functional  terms,  rather  than  abstract,  unusual  or  arbitrary  forms,  or  in  forms  requiring 
interpretation)  (pp.  A2-3) 

Vicente  (1990,  1997)  and  others  (Flach  and  Dominguez,  1995)  criticized  the  “user-centered  perspectives”  and 
disagreed  with  the  notion  that  the  plant  representations  provided  by  displays  must  be  compatible  with  the 
operator’s  mental  model  of  the  plant.  Vicente  argues  that  is  because  the  models  are  incomplete  and  sometimes 
incorrect  (as  was  discussed  in  Section  5.1).  Thus,  Vicente  asks  “what  good  is  it  having  the  computer  be  compatible 
with  the  agent’s  model  if  that  model  does  not  correspond  to  reality?”  (1990,  p.  496). 

An  example  of  the  failures  of  user-centered  design  is  the  use  of  operators  to  support  interface  design.  Vicente 
(1997)  gives  an  example  of  a  nuclear  plant  HSI  that  was  designed  with  the  aid  of  an  “exceptional  operator.”  When 
the  design  was  shown  to  other  operators,  it  was  not  accepted.  Vicente  concluded  that  “no  two  operators  seemed  to 
have  the  same  mental  model  of  the  plant”  and  that  “the  control  room  in  question  had  to  be  redesigned  to  reflect 
better  the  way  in  which  the  plant  actually  worked”  (p.  3).  Instead  Vicente  argues  that  process  control  operators 
exist  in  a  correspondence-driven  domain  where  there  is  an  external  reality  in  which  the  operator  must  exist.  This 
is  in  contrast  to  a  coherence-driven  domain,  where  the  human  and  the  computer  system  represent  a  closed  world 
and  need  not  reflect  external  constraints.  Rather  than  reflect  the  operator’s  model,  the  display  should  reflect  the 
actual  constraints  of  the  process  being  controlled.  Vicente  refers  to  this  as  the  “ecological  compatibility  principle.” 

We  feel  the  criticism  of  the  user-centered  perspective  takes  the  principles  out  of  context.  HSI  design  is  part  of  a 
systems  engineering  process  that  attempts  to  capture  the  functions,  relationships,  and  constraints  as  part  of 
functional  requirements  analysis  and  task  analysis.  Further,  the  systems  approach  derives  training  requirements 
that  are  meant  to  provide  operators  with  a  concept  of  operations  that  includes  an  understanding  of  the  system. 

Thus,  for  example,  the  NUREG-0700  principle  identified  above  is  part  of  a  greater  process  described  in  NUREG- 
0711.  This  context  was  recognized  by  Pejtersen  and  Rasmussen  (1997)  as  well,  who  identify  one  of  the  questions 
to  be  addressed  in  the  evaluation  of  the  complex  system  as  “Does  the  presentation  match  the  user’s  cognitive 
processes  and  mental  models?”  This  is  not  the  only  question,  and  Pejtersen  and  Rasmussen  suggest  that  questions 
on  the  content  of  the  information  in  representations  are  better  addressed  by  analytical  evaluation  methods,  while 
questions  addressing  form  are  better  addressed  using  empirical  evaluation  methods.  In  fact,  other  authors  view 
EID  as  a  movement  toward  user-centered  design.  Reising  and  Sanderson  (1996)  stated  that  EID  is  “in  a  position  to 
bring  about  a  profoundly  user-centered  approach  to  system  design  in  which  the  ultimate  information  needs  of 
human  controllers  drive  the  engineering  agenda  of  sensors  and  instrumentation  in  a  feed  forward  manner”  (p. 

296). 

The  point  of  user-centered  principles  (in  their  full  design  context)  is  that  once  the  systems  analyses  are  completed 
and  operators  are  trained,  the  HSI  should  be  consistent  with  human  cognitive  and  physical  constraints  and  should 
be  compatible  with  the  training  and  experience  used  to  develop  operators’  expertise  with  the  system. 

Thus,  we  do  not  disagree  with  Vicente’s  ecological  compatibility  principle.  The  HSI  must  reflect  the  reality  of  the 
system  being  operated.  Display  design  should  reflect  the  systems  engineering  process.  If  it  does,  the  process 
should  not  lead  to  displays  based  on  faulty  mental  models  of  individuals.  For  the  NPP  CR  design  discussed  above, 
the  failure  is  not  in  the  user-centered  aspects  of  the  design,  but  in  the  design  process  at  large.  Operators’  input 
should  be  sought  after  the  systems  analyses  to  help  develop  representations  that  link  display  forms  and  functions  to 
the  way  the  plant  works.  The  design  process  should  have  allowed  for  display  requirements  based  on  systems 
analyses  to  have  been  developed  and  for  prototyping  and  testing  alternatives  of  design  early  in  the  process  (see 
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NUREG-071 1).  HSIs  should  not  be  developed  based  on  the  input  of  a  single  operator  without  the  checks  and 
balances  offered  by  the  full  design  process. 

In  fact,  the  representation  of  a  system  as  complex  as  a  NPP  will  ultimately  be  based  on  many  sources  of  human 
expertise,  including  operators,  engineers,  and  physicists.  All  of  these  experts  are  likely  to  have  some  gaps  in  their 
mental  models. 

An  extreme  expression  of  the  ecological  compatibility  principle  was  reflected  in  a  paper  by  Lindsey  and  Staffon 
(1988;  from  Hansen,  1995).  They  developed  a  modified  Rankine  cycle  display  and  concluded  that  operators  no 
longer  needed  to  construct  a  mental  model  of  the  process  because  the  display  accurately  depicts  the  plant  state. 
While  displays  should  certainly  provide  accurate  representations  of  the  systems  they  represent,  the  relationship 
between  them  also  must  be  consistent  with  operator  models  of  the  system.  Consider  the  implications  of  violating  a 
principle,  such  as  the  User-Model  Compatibility.  There  are  ample  data  to  show  that  when  displays  are  not 
consistent  with  operator  models  and  expectations  are  not  well  integrated  into  task  environments,  they  will  not  be 
used,  or  even  worse,  they  will  be  misunderstood. 

Operators  interpret  information  using  their  mental  models,  which  provide  meaning  and  coherence  to  information. 
Ignoring  this  information  processing  reality  is  as  serious  as  ignoring  the  constraints  of  the  system.  Both  need  to  be 
equally  addressed  designing  an  effective  HSI.  Our  point  is  consistent  with  the  notion  of  the  cognitive  system  triad 
of  Woods  and  Roth  (1988),  which  includes  the  world  to  be  acted  on,  the  agents  who  will  act,  and  the  representation 
through  which  the  agent  acts.  All  three  are  essential  parts  of  the  overall  system. 

(8)  Training  and  Qualification  Implications 

Christoffersen  et  al.  (1995)  concluded  that  “to  experience  the  benefits  of  EID,  it  seems  likely  that  operators  need  to 
be  trained  to  think  functionally  rather  than  procedurally.  It  would  seem  that  this  would  require  a  fundamental  shift 
in  NPP  operation  philosophy.. .it  may  be  that  operators  have  to  possess  certain  types  of  cognitive  characteristics  that 
may  not  be  considered  in  the  traditional  selection  process  in  the  nuclear  industry”  (p.  143).  Concern  also  was 
expressed  that  EID  interfaces  may  inhibit  long-term  learning  and  retention  (Wickens,  1992). 

Also,  for  any  such  display,  where  the  information  is  sophisticated  and  a  large  amount  is  consolidated  into  one  or  a 
few  figures,  the  requirements  for  training  become  very  important.  One  needs  to  understand  all  of  the  aspects  of  the 
display  and  how  it  reacts  to  various  operational  transients,  accidents,  and  failure  of  instruments.  Finally,  the  long¬ 
term  effects  of  EID-type  displays  on  operators’  performance  and  strategies  are  unknown. 

(9)  Operator  Acceptance 

The  issue  of  operator  acceptance  of  a  new  and  different  type  of  display  (such  as  an  EID  display)  also  is  important, 
as  shown  by  the  operators’  comments  during  experiments.  One  interesting  idea  that  was  noted  during  the  Rankine 
cycle  experiments  is  introducing  the  displays  initially  in  training,  and  then  perhaps  gradually  introducing  them 
into  the  plant. 

(10)  Effect  of  Instrumentation  Failures  on  Interpreting  Displays 

The  effects  of  failures  of  instrumentation  on  EID  displays  is  a  potentially  significant  problem  that  needs  to  be 
addressed.  There  are  several  subissues  associated  with  this  problem: 

•  Can  a  failure  of  instrumentation  be  detected  by  operators? 
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•  Can  failures  result  in  representations  that  are  interpreted  by  operators  as  real  process  failures,  and  perhaps 
more  importantly,  can  real  process  failures  be  misinterpreted  as  instrument  failures? 

•  If  operators  detect  a  failure,  should  they  suspend  use  of  the  display? 

•  Since  the  display  integrates  many  parameters  into  a  single  display,  what  is  the  effect  of  its  loss  on  operations, 
and  how  effectively  can  operators  switch  to  backup  displays? 

Many  designers  have  implemented  advanced  features  that  address  this  concern  to  some  extent.  For  example,  the 
concepts  of  redundancy  and  diversity  in  instrumentation,  coupled  with  automatic  parameter  validation  and 
appropriate  notification  of  operators  when  instruments  fail  their  validation,  have  been  shown  to  be  powerful. 

5.4  Research  on  Information  Representation 

As  discussed  in  the  previous  section,  advances  in  the  analysis  of  information  requirements  have  led  to  potentially 
enhanced  methods  for  identifying  levels  of  information  about  a  plant  that  vary  along  A-A  dimensions.  However,  it 
is  not  adequate  to  identify  the  information  needed  and  simply  make  it  available  in  the  CR  (Woods  and  Elias,  1988; 
Woods  and  Roth,  1988).  Data,  in  and  of  themselves,  have  no  inherent  meaning.  Meaning  is  acquired  when  data 
are  put  in  context.  Thus,  “data  availability”  is  distinguished  from  “information  extraction”  Further,  this  context 
should  be  reflected  in  the  representation  of  the  information.  An  intelligent  display  system  would  “create  and 
manipulate  the  representation  through  which  the  domain  problem  solver  sees  the  world”  (Woods  and  Elias,  1988, 
p.  1352).  Representation  addresses  the  manner  in  which  the  information  needed  by  the  operator  is  displayed. 

Conventional  CRs  present  individual  parameters  on  analog  displays  as  separate  pieces  of  information.  The 
displays  are  usually  grouped  logically.  However,  integration  and  interpretation  of  data  is  mainly  accomplished  by 
the  operators  based  on  training  and  experience.  When  computer-based  display  technology  was  first  introduced  into 
CRs,  the  displays  tended  to  follow  well-known  information  presentation  formats  showing  single  parameter  values, 
such  as  digital  displays,  multiple  parameter  values,  such  as  bar  charts,  and  simple  relationships  between  variables, 
such  as  trend  graphs  vs.  time.  The  first  attempts  at  making  displays  ecological  linked  individual  parameter 
displays  with  mimic  lines  to  reveal  their  structure  with  respect  to  the  plant  and  process  they  represented.  While 
this  approach  was  a  common  modification  of  a  NPP  CR  following  the  TMI  event,  the  tradition  has  continued  in  the 
first  generation  of  digital  CRs,  which  make  extensive  use  of  displays  to  show  plant  components  and  systems  at  low 
levels  of  abstraction;  i.e.,  individual  components  may  be  represented  by  icons.  Such  displays  can  be  based  on  a 
functional  layout  of  the  plant,  using  the  format  of  simplified  piping  and  instrumentation  diagrams  (P&IDs). 

Comparisons  between  some  of  the  above  simple  representational  formats  revealed  differences  in  performance 
which  appear  to  be  task  specific.  For  example,  Hollands  and  Spence  (1992)  found  that  change  was  judged  more 
quickly  and  accurately  with  line  and  bar  graphs  than  with  tiered  bar  graphs  and  pie  charts,  while  the  opposite 
results  were  obtained  for  proportion  judgements.  Research  examining  the  effects  of  basic  display  representations 
on  specific  tasks  is  still  the  subject  of  ongoing  research  (e.g.,  Gillan  et  al.,  1998;  Hink  et  al.,  1996;  Hollands  and 
Dyre,  1997;  Hollands  and  Spence,  1997;  Meyer,  1997;  Meyer,  Gopher,  and  Levy,  1997;  Meyer,  Shinar,  and  Leiser, 
1997;  Shamo  etal.,  1996). 

The  guidance  on  design  review  contained  in  NUREG-0700  addresses  these  basic  representational  formats  fairly 
well.  Representational  formats  based  on  graphical  features  that  are  tied  to  perceptual  processes,  such  as  polar 
graphic  displays,  and  formats  using  graphic  forms  that  reveal  higher-level  plant  information,  such  as  Rankine 
cycle  displays  are  not  addressed  in  sufficient  detail.  Thus,  the  main  focus  of  this  section  is  on  representation 
formats  that  attempt  to  reveal  plant  information  by  making  better  use  of  human  perceptual  and  cognitive  processes. 
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The  term  representational  aiding  has  been  used  to  refer  to  using  the  capability  of  computer-generated  displays  to 
create  novel  representations  of  a  system  that  can  be  manipulated  to  support  operator’s  decision  making  (Woods  and 
Roth,  1988). 

Sections  5.4.1  and  5.4.2,  respectively,  discuss  the  theory  and  research  directed  to  representational  aiding.  Section 
5.4.3  gives  the  conclusions. 

5.4.1  Theory  of  Representational  Aiding 

Direct  Perception  and  Representational  Mapping 

As  noted  earlier,  NPPs  generate  a  great  quantity  of  data  that,  at  times,  can  overload  the  operator.  To  lower  the 
workload  associated  with  extracting  meaningful  information  from  data,  efforts  were  undertaken  to  support  the  task 
of  integrating  data  into  more  meaningful  units  of  information.  Within  the  context  of  representational  aiding,  the 
goal  of  display  design  is  “...to  map  the  domain  semantics  (low-level  data,  high-level  constraints,  and  relevant 
performance  goals)  into  the  appearance  and  dynamic  behavior  of  a  graphic  display  so  that  this  information  is 
readily  available  (easily  extracted  or  decoded  by  the  user)”  (Bennett  et  al.,  1993,  p.  73). 

The  success  of  representational  aids  is  a  function  of  the  characteristics  of  the  system,  the  tasks  to  be  performed,  and 
human  information  processing  (Bennett,  1 992).  Woods  and  Roth  (1988)  suggested  that  displays  can  be 
conceptualized  as  involving  a  cognitive  system  triad  that  includes  the  world  to  be  acted  on,  the  agent  who  will  act 
on  the  world,  and  the  representation  through  which  the  agent  acts  on  the  world.  All  three  are  essential  parts  of  the 
overall  system.  The  representation  involves  the  following  two  types  of  mapping  (Bennett  et  al.,  1997): 

•  Correspondence  map  between  the  properties  and  characteristics  of  the  system  to  be  represented  and  the 
features  in  the  representation  (how  well  the  display  communicates  meaningful  information  about  the  plant  that 
is  needed  by  operators) 

•  Coherence  map  between  the  features  in  the  representation  and  the  physical  and  cognitive  characteristics  of  the 
operator  (how  comprehensible  the  representation  is  to  the  operator) 

The  systems  engineering  approach  (see  Section  5.1)  and  abstraction  hierarchy  (see  Section  5.3)  provide  approaches 
that  address  half  of  the  correspondence  issue  -  the  identification  of  system  functions  and  constraints  that  need  to  be 
represented. 

In  the  previous  section,  EIDs  were  discussed  in  terms  of  Gibson’s  ecological  psychology  and  its  focus  on  the  work 
domain  or  system  to  be  represented.  Two  aspects  of  Gibson’s  approach  to  perception  are  important  to  representing 
information  about  the  system:  the  concepts  of  direct  perception  and  affordances  (Flach,  1989;  Flach  and  Hancock, 
1992).  Direct  perception  means  that  the  information  presented  in  the  display  has  immediate  meaning  within  the 
context  of  the  operator’s  current  information  needs,  including  task  requirements  and  system  status.  There  is  little 
need  for  the  operator  to  analyze  and  interpret  direct  perception  displays  (see  Flach,  1988,  for  a  discussion  of  the 
use  of  the  term  “direct”  in  HSI  design).  Affordances  are  the  features  of  an  interface  that  suggest  an  interpretation 
or  a  response  by  the  crew.  They  are  a  product  of  the  interaction  between  the  crew  and  the  interface  within  the 
context  of  the  ongoing  tasks  of  the  crew.  This  is  an  extension  of  what  Gibson  meant  by  direct  perception  of  the 
physical  features  of  the  perceptual  environment.  Direct  perception  is  made  possible  by  a  one-to-one  mapping  of 
affordances  and  domain  constraints. 
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Thus,  the  degree  to  which  a  display  supports  operator’s  performance  is  a  function  of  both  the  quality  of  the 
mapping  between  the  emergent  display  features  perceived  by  the  operator  and  the  relationships  and  constraints  of 
the  system.  Bennett  et  al.  (1997)  stated  that 

Problem  solving  can  be  critically  influenced  by  the  nature  of  visual  representations.  Building  effective 
representations  requires  designers  to  go  beyond  the  simple  psychophysical  questions  of  data  availability  to  the  more 
complex  questions  of  information  availability,  where  information  refers  to  the  specification  of  domain  constraints  and 
boundary  conditions.  This  specification  depends  both  on  the  mapping  from  display  to  human  (i.e.,  coherence)  and  on 
the  mapping  from  display  to  domain  (i.e.,  correspondence),  (p.  682) 

Efforts  to  create  direct  links  between  graphical  forms  and  operator  actions  are  discussed  below  (these  links  are  the 
affordances).  Two  basic  approaches  are  discussed:  the  information  relationships  approach  and  the  proximity- 
compatibility  approach.  The  primary  focus  of  the  information  relationships  approach  is  on  the  mapping  between 
display  characteristics  and  characteristics  of  the  system  being  represented,  while  the  primary  focus  of  the 
proximity-compatibility  approach  is  the  mapping  between  display  characteristics  and  the  characteristics  of  the 
tasks  being  performed. 

Information  Relationships:  Representing  Separable,  Integral,  and  Configure!  Relationships 

One  attempt  to  accomplish  the  objective  of  direct  perception  is  based  on  the  mapping  of  low-level  perceptual 
processes  to  the  relationships  between  plant  information.  Bennett  and  Flach  (1992)  defined  three  potential 
relationships  between  data  that  can  be  represented  in  displays:  separable,  integral,  and  configural.  Separable 
relationships  are  those  that  can  be  showed  by  individual  data  points.  They  are  characterized  by  a  lack  of 
interaction  among  the  data  dimensions;  i.e.,  each  dimension  retains  its  unique  perceptual  identity.  Integral 
relationships  are  defined  by  a  strong  interaction  among  data  dimensions  such  that  the  unique  identities  of 
individual  dimensions  are  lost.  When  such  a  relationship  exists,  a  change  in  one  dimension  necessarily  produces  a 
change  in  a  second  dimension.  Bennett  and  Flach  used  the  example  of  color  perception,  which  is  a  function  of  the 
dimensions  of  hue  and  brightness.  In  color  perception,  the  perception  of  each  individual  dimension  is  lost  in  the 
integration.  Configural  relationships  refer  to  an  intermediate  level  of  interaction  between  dimensions,  where  each 
maintains  its  unique  identity,  but  new  emergent  properties  are  created  as  a  result  of  the  interaction  between  the 
dimensions. 

These  three  types  of  relationships  are  associated  with  three  generic  types  of  displays.  An  important  goal  in  display 
design  is  to  provide  the  most  appropriate  type  based  on  how  the  operator  must  use  information  about  the  domain. 

Separable  displays  are  representations  where  each  process  parameter  is  presented  individually  and  no  relationships 
between  the  parameters  are  provided  by  the  representation  itself.  The  key  aspect  of  separable  displays  is  not  that 
individual  parameters  are  presented,  but  that  no  interaction  or  relationship  between  them  is  perceived.  There  may 
be,  in  fact,  a  relationship  between  the  parameters,  but  the  separable  displays  do  not  show  it.  The  studies  on  the 
Rankine  cycle  display,  discussed  in  Section  5.3,  used  a  separable  display  -  a  set  of  individual  meters  -  as  a  basis  of 
comparison  (see  Figure  5.12). 

Integral  displays  show  the  integration  of  information  in  such  a  way  that  the  individual  parameters  used  to  generate 
the  display  are  not  represented  in  it.  A  display  that  provides  information  on  system  status  by  the  appearance  of  an 
icon  is  an  example.  The  icon  may  change  appearance  based  on  the  computation  of  lower-level  parameters,  but 
these  parameters  are  not  presented.  When  decisions  are  based  on  the  integrated  information,  the  concept  of 
integral  display  suggests  that  information  processing  is  supported  by  integrating  information  into  a  single  object 
(Kahneman  and  Triesman,  1984).  Integral  displays  are  thought  to  support  direct  perception  by  capitalizing  on  the 
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abilities  of  human  pattern  recognition,  and  thereby  reduce  demand  on  working  memory  associated  with 
simultaneous  consideration  of  many  individual  variables. 

Configural  displays  provide  a  combination  of  the  other  two  types.  Information  about  the  basic  parameters  is 
available;  however,  information  on  their  relationships  becomes  evident  as  features  emerge  from  the  representation. 
An  emergent  feature  is  a  “high-level,  global  perceptual  feature  that  is  produced  by  the  interactions  among 
individual  parts  or  graphical  elements  of  a  display  (e.g.,  lines,  contours,  and  shapes)”  (Bennett  et  al.,  1993,  p.  73). 
Configural  displays  often  use  simple  graphic  forms,  such  as  a  polygon.  Information  that  could  be  presented  by 
separate  display  formats  is  integrated  into  a  single  format  in  which  each  of  the  separate  pieces  of  information  is 
represented,  for  example,  by  the  distance  of  a  polygon’s  vertex  from  its  center.  In  addition,  the  geometric  shape  of 
the  polygon  provides  a  high-level  summary  (the  emergent  feature)  (Buttigieg  et  al.,  1988;  Sanderson  et  al.,  1 989). 

The  SPDS  display  described  in  Section  4  is  an  example  of  a  configural  display  (Woods  et  al.,  1981).  By  glancing 
at  the  shape  of  the  polygon,  the  operator  can  readily  see  whether  the  plant  is  in  a  safe  or  unsafe  mode.  However, 
information  about  the  individual  parameter  values  also  is  still  available.  Such  displays  are  thought  to  enhance 
parallel  processing  (lowering  cognitive  workload),  enable  operators  to  better  understand  the  relationships  between 
display  elements,  and  ultimately  lead  to  a  more  rapid  and  accurate  awareness  of  the  situation  (Bennett  and  Flach, 

1 992;  Flach  and  Bennett,  1 992).  The  extraction  of  information  for  high-level  task  requirements  is  supported  by  the 
emergent  features  produced  by  a  configural  display. 

Another  example  of  a  configural  display  is  the  Rankine  cycle  graphic  discussed  in  Section  5.3  (see  Figure  5.1 1). 
The  form  of  the  graphic  indicates  overall  performance,  and  the  operator  can  obtain  specific  parameter  information 
if  needed.  However,  in  this  Rankine  display,  the  parameters  would  have  to  be  estimated  from  the  figure,  and  the 
scales  are  large.  This  almost  makes  it  into  an  integral  display  (without  parameter  values),  since  the  operators 
would  really  need  to  refer  to  separate  indicators  to  obtain  useful  specific  information  on  pressure,  temperature,  or 
levels. 

Emergent  features  can  be  formed  by  purely  separable  displays;  i.e.,  where  no  specific  graphic  is  incorporated  into 
the  display  (Wickens  and  Carswell,  1995).  A  systematic  pattern  formed  by  a  series  of  bars  on  a  bar  chart  or  the 
parallel  appearance  of  a  set  of  analog  meters  may  form  an  emergent  feature  (Figure  5.17).  Operators  may,  for 
example,  be  able  to  glance  at  a  row  of  meters  and  be  able  to  determine  that  all  systems  are  properly  functioning 
because  the  needles  are  all  pointed  in  the  same  direction.  Similarly,  the  blackboard  (or  darkboard)  concept  in 
alarm  system  design  is  based  on  an  emergent  feature  (O’Hara  et  al.,  1994).  The  darkboard  concept  means  that 
when  all  alarmed  parameters  are  within  normal  operating  range,  no  lighted  alarms  can  be  seen  (on  a  conventional 
board)  or  no  messages  appear  on  alarm  VDUs.  When  the  alarm  system  is  designed  in  this  way,  the  operators  can 
glance  at  the  alarm  display  and  determine  the  general  acceptability  of  overall  plant  performance.  The  emergent 
feature  is  effective  if  it  conveys  high-level  information  without  requiring  operators  to  analyze  the  individual 
parameters. 

The  degree  to  which  the  individual  and  emergent  features  are  perceptually  salient  to  the  observer  is  important  in 
allocating  attention  to  the  elements  of  the  display.  The  design  goal  is  to  map  meaningful  aspects  of  the  operator’s 
task  information  requirements  to  the  visual  characteristics  of  the  display  (the  display  grammar,  as  discussed  in 
Section  4).  Significant  aspects  of  domain  structure  and  behavior  must  be  mapped  to  the  characteristics  of  the 
representation  so  that  the  domain’s  semantics  are  directly  visible  to  the  operator  (Woods  and  Roth,  1988).  Issues 
that  need  to  be  addressed  include  selecting,  scaling,  and  assigning  lower-order  data  to  characteristics  of  the 
representation.  The  challenge  is  to  set  up  a  mapping  so  that  the  task-meaningful  semantics  are  directly  visible  to 
the  observer  and  that  both  normal  and  disturbance  conditions  are  clearly  perceived. 
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(a)  Emergent  features  showing  normal  operating  conditions 


(b)  Same  normal  operating  conditions  without  emergent  features 

Figure  5.17  Separable  Display  With  and  Without  Emergent  Features 

The  value  of  each  type  of  display  -  separable,  integral,  and  configural  -  may  depend  on  the  task  requirements 
(Bennett  and  Flach,  1992).  That  is,  if  operators  must  conduct  their  tasks  based  merely  on  individual  parameters, 
then  separable  displays  may  be  preferred.  If  they  depend  on  the  integration  of  individual  parameters,  then  integral 
displays  may  be  preferred.  If  performance  is  based  on  a  combination  of  the  two,  then  configural  displays  may  be 
the  best  solution  for  representation. 

Bennett  et  al.  (1997,  pp.  291-292)  identified  the  following  design  principles  based  on  information  relationships: 

(1)  Each  relevant  process  variable  should  be  represented  by  a  distinct  element  within  the  display.  If  precise 
information  about  this  variable  is  needed,  then  a  reference  scale  or  supplemental  digital  information  should  be 
provided. 

(2)  The  display  elements  should  be  organized  so  that  the  emergent  properties  (symmetries,  closure,  parallelism) 
that  arise  from  their  interaction  correspond  to  higher-order  constraints  within  the  process.  Thus,  when  process 
constraints  are  broken  (i.e.,  a  fault  occurs),  the  corresponding  geometric  constraints  also  are  broken  (the 
display  symmetry  is  broken). 

(3)  The  symmetries  within  the  display  should  be  nested  (from  global  to  local)  in  a  way  that  reflects  the 
hierarchical  structure  of  the  process.  High-order  process  constraints  (e.g.,  at  the  level  of  functional  purpose  or 
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abstract  function)  should  be  reflected  in  global  display  symmetries;  lower-order  process  constraints  (e.g., 
functional  organization)  should  be  reflected  in  local  display  symmetries. 

Proximity-Compatibility  Principle 

The  proximity-compatibility  principle  (PCP)  was  developed  to  guide  the  mapping  of  operator  task  characteristics 
and  display  design  (Carswell  and  Wickens,  1987;  Wickens  and  Carswell,  1995).  Task  proximity  and  display 
proximity  need  to  be  jointly  considered  when  designing  displays  to  support  task  performance.  The  task  and  display 
characteristics  are  described  in  more  detail  below. 

Task  proximity  can  be  described  by  several  dimensions,  including  the  following: 

•  Temporal  proximity  -  the  degree  to  which  two  tasks  must  be  performed  in  close  temporal  proximity  to  achieve 
a  goal 

•  Processing  proximity  -  the  overlap  between  the  information  processing  that  is  needed  to  process  different 
information 

•  Statistical  proximity  -  the  degree  of  covariation  between  information  (e.g.,  when  two  parameters  covary  or 
when  a  change  in  one  is  reflected  in  a  change  in  the  other,  the  relationship  has  high  statistical  proximity) 

•  Functional  proximity  -  the  similarity  of  objects  as  represented  in  the  operator’s  mental  model  (e.g.,  all 
parameters  describing  the  performance  of  a  single  component  have  high  functional  proximity) 

Three  general  categories  of  tasks  are  defined:  integrative,  nonintegrative  processing  of  similar  tasks,  and 
nonintegrative  processing  of  dissimilar  tasks.  Integrative  processing  tasks  require  operators  to  engage  in 
(1)  computational  integration,  i.e.,  combining  or  integrating  data  to  make  a  decision  or  take  an  action;  or  (2) 
boolean  integration,  i.e.,  determining  the  satisfaction  of  boolean  conditions  to  make  a  decision  or  take  an  action. 

In  both  cases,  operators  are  using  many  information  sources  to  arrive  at  a  few  decisions  or  to  take  a  few  actions  (a 
many  to  few  relationship). 

In  contrast  to  integrated  tasks,  nonintegrative  processing  of  dissimilar  tasks  is  characteristic  of  task  independence 
or  low  task  proximity.  There  is  little  similarity  or  interaction  between  the  information  or  processing  mechanisms 
required  by  two  tasks.  Nonintegrative  processing  of  similar  tasks  is  characteristic  of  tasks  with  moderate 
proximity. 

Display  proximity  addresses  how  perceptually  similar  two  displays  are  that  convey  information  on  the  same  task. 
There  are  several  dimensions  that  define  perceptual  proximity,  including  spatial  proximity,  chromatic  proximity, 
physical  dimensions  used  to  code  information  (such  as  length),  perceptual  coding  (such  as  digital  vs.  analog),  and 
geometric  (integrated  or  separate  displays)  (Bennett  et  al.,  1997).  Display  proximity  is  enhanced  by  (1)  arranging 
displays  close  together,  (2)  using  shared  information  channels,  (3)  adding  lines  to  connect  or  enclose  information 
channels,  (4)  giving  related  information  sources  the  same  codes,  such  as  color  and  orientation,  (5)  integrating 
information  so  it  appears  as  a  single  object  (object  integration),  and  (6)  configuring  homogenous  features  into  a 
new  pattern  (configuration). 

The  PCP  states  that  there  should  be  a  high  correlation  between  task  proximity  and  display  proximity.  High- 
proximity  tasks,  such  as  those  involved  in  process  fault  detection,  must  be  performed  in  a  highly  integrated 
manner.  These  types  of  tasks  are  conducted  better  using  high-proximity  displays,  e.g.,  by  presenting  data 
relationships  as  a  single  object  (an  integral  object  display),  because  many  plant  variables  must  be  considered 


NUREG/CR-6633 


5-58 


5  DEVELOPMENT  OF  TECHNICAL  BASIS 


simultaneously.  High  proximity  is  accomplished  by  high  integration  of  data.  Low-proximity  tasks  are  those  in 
which  the  operator  responds  by  focusing  on  individual  system  elements.  These  tasks  are  better  supported  by  low- 
proximity  displays,  such  as  a  series  of  bar  charts.  Essentially,  the  PCP  states  that  low-proximity  tasks  will  be  better 
performed  with  low-proximity  displays  and  vice  versa. 

The  basic  phenomena  that  account  for  high-proximity  display  support  of  high-proximity  tasks  are  reduced 
information  access  cost  (I AC),  object  integration,  and  emergent  features  (Wickens  and  Carswell,  1995;  Wickens, 
1986).  Reduced  IAC  is  accomplished  by  (1)  allowing  all  display  axes  to  be  within  foveal  vision,  thus  minimizing 
search  and  scanning  problems;  (2)  reducing  the  memory  load  associated  with  retaining  part  of  the  needed 
information  while  another  display  is  accessed  to  get  additional  needed  information;  and  (3)  reducing  the  mental 
effort  required  to  integrate  the  information.  The  more  separate  the  information,  the  more  IAC  is  involved. 

Emergent  features  and  object  integration  enable  perception  to  be  used  instead  of  mental  computation.  Emergent 
features,  however,  need  to  be  mapped  to  task-related  variables. 

These  same  phenomena  can  make  the  performance  of  low-proximity  tasks  more  difficult  because  high-proximity 
displays  will  create  clutter  on  the  screen  and  lead  to  confusion.  Close  spatial  proximity  can  make  it  difficult  to  find 
relevant  information  and  discriminate  it  from  other  information.  When  a  relevant  object  is  within  one  visual 
degree  of  an  irrelevant  object,  the  presence  of  one  has  a  negative  effect  on  the  processing  of  the  other  (and  will 
increase  IAC).  Conversely,  decreasing  proximity  of  the  display  makes  it  easier  to  focus  attention  on  a  single  object 
or  parameter,  and  thereby  supports  the  performance  of  low-proximity  task.  Thus,  the  PCP  places  emphasis  on  the 
mapping  of  display  characteristics  to  task  demands.  Compatibility  between  the  two  will  support  performance  while 
incompatibility  will  detract  from  performance. 

5.4.2  Research  on  the  Effectiveness  of  Representational  Aiding 

In  this  section,  we  consider  the  research  addressing  these  approaches  to  representational  aiding.  Where  possible, 
published  reviews  of  the  literature  were  used.  These  reviews  were  supplemented  with  individual  research  reports. 
For  discussion  of  the  studies  addressing  representational  aiding,  the  term  focused  tasks  refers  to  tasks  that  are 
independent  from  each  other  in  terms  of  requiring  separate  information,  and  the  term  integrated  tasks  is  used  to 
refer  to  tasks  that  require  integrating  information.  The  discussion  below  centers  around  two  main  questions: 

(1)  what  are  the  overall  effects  of  integral,  configural,  and  separable  displays  on  performance?  and  (2)  what  are  the 
specific  design  features  that  explain  these  effects? 

Bennett  and  Flach  (1992)  conducted  a  literature  review  of  studies  examining  the  relative  effects  of  configural 
graphical  displays  and  separable  displays  on  performance  of  integrated  and  focused  tasks.  They  grouped  the 
studies  by  test  methodology,  as  follows:  signal  detection,  multiple-cue  judgement,  retrospective  memory  probe, 
and  system  control. 

In  the  signal  detection  methodology,  a  participant  observes  a  display  and  detects  an  abnormality.  No  actual  system 
control  is  performed  by  the  participant.  The  results  of  studies  using  this  methodology  generally  showed  that  the 
configural  displays  led  to  better  detection  performance  than  separable  displays,  especially  in  integrated  tasks 
requiring  higher-level  information.  For  tasks  focused  on  lower-level  information,  the  results  were  either 
insignificant  or  favored  the  separable  displays. 

Studies  using  a  multiple  cue  judgement  methodology  require  the  participants  to  observe  a  display  and  make  a 
judgement  about  the  system’s  status.  The  results  were  generally  consistent  in  support  of  configural  displays  in 
integrated  tasks.  Bennett  and  Flach  found  only  one  study  using  focused  tasks  where  performance  was  better  with 
the  separable  display. 
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In  studies  using  the  retrospective  memory  probe  methodology,  participants  were  asked  to  recall  information  about  a 
system  based  upon  display  conditions.  Again,  participants  performed  better  with  configural  displays  for  integrated 
tasks,  but  the  display  differences  for  focused  tasks  were  insignificant. 

Finally,  in  the  system  control  method,  participants  monitor  and  control  a  dynamic  system.  Performance  measures 
reflecting  performance  goals  are  obtained.  Bennett  and  Flach  found  little  research  using  this  method  (see 
discussion  of  Bennett  et  aL,  1993,  below).  In  one  area,  the  results  from  multiaxis  tracking  tasks  supported  the  use 
of  configural  displays. 

In  summary,  the  review  showed  that  better  performance  was  found  with  configural  displays  when  the  task  was 
integrated,  and  therefore,  required  performance  based  on  abstract  integration  of  data.  For  focused  tasks,  where 
performance  required  responses  based  on  specific  aspects  of  the  data,  a  “mixed  pattern  of  results”  was  found 
(Bennett  and  Flach,  1992;  Bennett  et  al.,  1 997);  i.e.,  results  were  either  insignificant  or  favored  separable  displays. 

Barnes  and  Suantak  (1996)  evaluated  the  effects  of  configural  representations  of  complex  tactical  situations  in 
comparison  to  conventional  graphic  and  alphanumeric  formats  on  the  speed  and  accuracy  of  tactical  decisions. 
Seventeen  army  officers  made  situational  decisions  based  on  the  displays,  and  responses  were  faster  and  more 
accurate  with  the  configural  displays. 

Bennett  et  al.  (1993)  used  the  system  control  method  to  compare  the  performance  of  20  students  on  a  simulated 
manual  feedwater  control  task,  using  either  separable  or  configural  displays  in  two  experiments.  In  experiment  1, 
the  task  was  to  adjust  the  flow  rate  to  maintain  steam  generator  level.  Task  performance  was  the  primary 
dependent  variable.  No  significant  differences  between  the  displays  was  observed.  In  experiment  2,  using  the 
same  participants  and  control  tasks,  a  memory  task  was  included  to  examine  recall  for  focused  information  (e.g., 
rate  of  steam  flow)  or  integrated  information  (e.g.,  the  difference  between  steam  and  feedwater  flow).  To  ask  the 
memory  question,  the  simulation  was  stopped  and  computer  screens  went  blank.  After  the  memory  question  was 
answered,  the  simulation  was  resumed.  The  results  showed  that  there  was  a  significant  interaction  of  display  type 
and  question  type.  The  configural  display  significantly  improved  performance  on  the  integrated  information,  but 
not  on  the  focused  information.  Also,  while  the  configural  display  improved  memory  for  important  high-level 
information,  differences  in  actual  task  performance  were  not  significant.  This  result  may  parallel  those  of  Vicente 
and  colleagues  (discussed  in  Section  5.3),  where  the  configural  information  in  the  DURESS  EID  display  did  not 
greatly  affect  the  control  task  performance. 

Coury  and  Boulette  (1992)  compared  several  graphical  formats  and  found  that  a  task’s  time  constraint  was  an 
important  factor  affecting  the  display-task  relationships.  They  found  that  “different  representations  of  the  same 
system  data  can  have  a  profound  effect  on  performance”  (p.  707).  They  further  stated  that 

One  clear  conclusion  from  this  research  is  that  the  effects  of  time  stress  complicate  the  selection  of  display  formats 
for  state  identification  tasks.  Although  designers  of  operator  interfaces  may  be  tempted  to  select  the  polygon  display 
because  of  its  apparent  superiority  under  the  most  severe  time  constraints,  such  a  choice  would  ignore  a  number  of 
important  trade-offs  in  performance.  Consider  changes  in  processing  strategy.  Although  overall  performance  with 
the  polygon  display  was  good,  the  digital  display  was  superior  in  many  situations,  (p.  723) 

Hurts  (1996)  investigated  memory  for  configural  information.  Twelve  undergraduate  students  responded  to 
questions  about  interpretation  of  graphed  information  using  either  separable  graphs  or  configural  graphs. 
Dependent  measures  were  time  and  accuracy.  The  results  revealed  that  while  configural  graphs  were  superior  to 
separable  graph  displays  when  the  graphs  were  present,  the  effect  was  reversed  when  retrieval  from  memory  was 
required  to  respond  to  questions  on  lower-level  information.  That  is,  performance  on  the  memory  test  was  superior 
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with  separable  graphs.  They  also  found  that  the  effect  was  lessened  when  there  was  greater  data-to-graph 
compatibility. 

While  integral  and  configural  displays  appear  to  support  some  aspects  of  integrated  task  performance,  some 
questions  remain.  What  accounts  for  the  effects,  i.e.,  is  it  that  the  information  is  integrated  into  objects  displays, 
such  as  polygons,  or  is  it  the  emergent  features?  Further,  how  might  performance  of  focused  tasks  be  better 
supported  in  higher-level  displays?  These  issues  are  discussed  below. 

Flach  and  colleagues  addressed  the  ‘object  vs.  emergent  feature'  aspect  of  displays  using  a  fault  detection  task 
(Buttigieg  et  al.,  1988;  Sanderson  et  al.,  1989).  Fault  detection  is  a  high-proximity  or  integrated  task  because 
many  variables  must  be  simultaneously  considered.  In  such  a  situation,  the  PCP  predicts  that  a  high-proximity 
display  will  lead  to  superior  performance.  An  integral  display  where  all  variables  are  integrated  into  a  single 
object  is  a  high-proximity  display.  The  emergent  features  approach  would  predict  that  it  is  not  the  object  per  se 
that  will  lead  to  superior  performance,  but  the  salience  of  emergent  features  even  if  the  individual  displays  are 
separate. 

To  compare  these  predictions,  participants  engaged  in  a  fault  detection  task  for  a  simple  process  consisting  of  two 
inputs  and  one  output.  Four  displays  were  used.  One  was  an  integral  display  where  the  inputs  and  outputs  were 
integrated  into  a  triangle  display.  The  other  three  displays  were  separate  bar  charts,  which  were  arranged  in 
different  ways  to  vary  the  salience  of  the  emergent  feature  created  by  the  three  bar  charts.  Reaction  time  and  false 
alarms  were  the  dependent  variables.  Contrary  to  the  proximity-compatibility  principle,  performance  was  better  on 
separate  bar  chart  displays  than  with  an  integrated  object  display.  The  bar  chart  associated  with  the  best 
performance  had  the  most  salient  emergent  feature  (linearity  of  the  bar  chart  arrangement). 

Thus,  it  may  not  be  the  integration  of  information  into  an  object  that  supports  performance,  but  rather  the  emergent 
features.  The  failure  of  integral  object  displays  to  support  focused  tasks  was  observed  as  well.  Montgomery  and 
Bivona  (1997)  compared  bar  charts,  two  types  of  deviation  barcharts,  and  an  integrated  object  display  for  detecting 
differences  in  the  temporal  variability  of  information  sources.  Fourteen  participants  observed  the  displays  and 
were  asked  to  decide  which  of  two  sources  was  least  variable.  Accuracy  was  the  dependent  measure.  The 
magnitude  of  the  difference  was  varied.  The  deviation  bar  chart  resulted  in  the  best  performance;  the  integral  and 
standard  bar  chart  displayed  were  the  worst.  The  authors  concluded  that  the  deviation  displays  were  more 
compatible  with  the  task,  which  required  the  source  variability  to  be  considered  separately. 

Thus,  focused  tasks  are  not  supported  by  integral  object  displays  and  may  not  be  supported  well  by  configural 
displays.  Bennett  et  al.  (1996)  addressed  methods  for  enhancing  the  extraction  of  lower-level  information  from 
configural  displays.  Configural  displays  with  and  without  several  enhancements  were  compared:  (1)  color  coding, 
layering,  and  separation;  (2)  tickmarks  and  gridlines,  (3)  bar  graphs  and  extenders,  and  (4)  digital  values.  These 
enhancements  were  systematically  included  and  excluded.  Eight  students  participated  in  the  experiments  and 
performed  a  simulated  process  control  task.  The  experimental  sessions  were  interrupted  at  random  intervals  and 
participants  were  asked  either  focused  memory  probes  (addressing  lower-level  information)  or  integration  memory 
probes  (addressing  higher-level  information).  Latency  and  accuracy  were  the  dependent  variables.  The  results 
showed  that  the  inclusion  of  digital  values  was  associated  with  the  greatest  increase  in  performance.  Another 
positive  enhancement  was  the  use  of  tickmarks  on  the  display  axes  and  gridlines  that  extended  the  scale 
information  into  the  display  area.  Thus,  configural  displays  can  be  designed  to  better  support  focused  tasks. 

Hansen  (1995)  also  examined  the  enhancement  of  configural  displays.  The  evaluation  included  the  degree  of 
configurality  (configural  or  separable),  presence  of  digital  information  (presented  or  not),  and  presence  of  time 
history  information  (presented  or  not).  Configural  and  separable  displays  were  compared  with  and  without  digital 
and  time  information.  The  experimental  task  was  to  monitor  eight  variables  to  determine  an  “event”  defined  by  a 
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decrease  in  all  eight  Measures  of  reaction  time,  event  hit  rate,  and  false  alarm  rates  were  obtained.  The  displays 
were  compared  in  two  experiments.  In  the  first,  the  rate  of  change  of  the  event  was  either  slow  or  rapid.  In 
experiment  2,  the  signal-to-noise  ratio  was  varied  between  noisy  and  not.  Each  experiment  had  1 1  participants, 
graduate  students,  in  a  within-subjects  design.  Overall,  the  results  showed  that  configural  displays  were  better  than 
separable  displays  for  event  hit  rate,  but  led  to  higher  rates  of  false  alarms.  The  addition  of  time  information  was 
found  most  useful  in  noisy  events.  In  an  analysis  of  the  combined  results  of  the  two  studies,  providing  digital 
information  led  to  faster  response  time,  but  there  was  no  effect  on  hit  or  false  alarm  rates.  The  digital  information 
provided  a  means  of  responding  when  the  graphic  display  began  to  indicate  the  event.  At  times,  digital 
information  interfered  with  the  emergent  features  of  the  integral  displays.  Hansen  (1995)  concluded  that  there  was 
much  support  for  the  observation  that 

...graphical  representations  can  be  effective  for  communicating  information  for  controlling  dynamic  systems. 

However,  there  is  less  agreement  as  to  the  dimensions  that  distinguish  effective  graphical  representations  from 
ineffective  ones.  Experimental  comparisons  of  display  formats  varying  in  degree  of  configural  ity,  inclusion  of 
supplemental  digital  information,  and  inclusion  of  explicit  time  dimensions  were  found  to  have  significant  effects  on 
the  detection  of  temporal  events  and  the  ability  to  detect  trends.  For  example,  it  was  observed  in  some  conditions  the 
inclusion  of  digital  information  in  an  integral  graphical  display  can  detract  from  its  emergent  features,  (p.  539) 

Several  studies  focused  on  polygon  displays.  Woods  et  al.  (1981,  1982)  compared  a  safety  panel  composed  of  a 
configural  polar  graphic  display  (see  Section  4)  and  a  plant  mimic  display  to  one  composed  of  trend  graphs  of 
safety  parameters.  Eight  crews  of  professional  operators  experienced  transients  on  a  NPP  simulator  during  their 
normal  training  sessions.  No  quantitative  comparisons  of  the  displays  were  performed.  In  a  qualitative  analysis, 
the  trend  display  did  not  support  operator  decision  making  because  of  implementation  problems,  e.g.,  the  update  • 
rate  was  too  slow.  The  polar  graphic  display  gave  a  useful  overview  of  the  plant’s  safety  status,  but  the  authors  felt 
that  lack  of  familiarity  limited  its  use.  The  mimic  display  was  the  most  effective  and  frequently  used  display. 

Other  studies  found  an  effect  of  polygons  on  performance.  Performance  on  a  fault  detection  task  was  found  to  be 
better  with  the  polygon  display  (Hughes  and  MacRae,  1994).  Greaney  and  MacRae  (1996)  evaluated  the  effect  of 
polygon  displays  on  fault  diagnosis.  As  the  number  of  vertices  increased,  the  reaction  time  for  diagnosis  increased. 
Reaction  time  also  increased  as  a  function  of  polygon  regularity,  that  is,  the  more  irregular  the  display,  the  longer 
it  took  to  diagnose  a  fault.  Thus,  such  displays  may  be  of  more  value  for  indicating  higher-level  information  than 
for  the  display  of  individual  parameters  that  must  be  assessed  separately. 

Green  and  Logie  (1996)  examined  the  use  of  polygon  displays  to  indicate  the  physiological  status  of  a  patient  in 
intensive  care.  Polygons  of  eight  to  ten  sides  were  developed.  They  found  that  information  on  status  could  quickly 
be  derived  from  the  displays  and  that  performance  was  better  for  simpler  polygons.  They  also  observed  that 
physicians  were  more  attentive  to  information  reflected  by  the  top  of  the  polygon;  that  is,  patient  information 
represented  by  the  upper  part  of  the  polygon  was  more  salient.  In  another  experiment,  polygons  were  developed  to 
indicate  specific  medical  conditions  that  were  typically  observed  in  intensive  care  units.  Physicians  were  readily 
able  to  use  these  displays  for  diagnosing  the  nature  and  severity  of  the  conditions. 

Pionek  et  al.  (1997)  investigated  the  effects  of  polygon  characteristics  on  reaction  time.  The  characteristics  were 
color  (color  vs.  no  color),  size  (large  vs.  small,  30%  of  large),  and  number  of  vertices  (6  or  12).  Fourteen 
engineering  students  viewed  the  polygon  displays  to  identify  the  presence  of  an  abnormal  state  using  reaction  time 
as  a  measure.  The  results  showed  that  color  and  size  were  significant,  as  was  the  interaction  of  color  and  number 
of  vertices. 

Coury  et  al.  (1989)  found  that,  for  fault  diagnosis,  a  polygon  display  was  worse  than  a  bar  chart  display,  but  was 
superior  to  a  digital  display.  Casey  (1986)  also  found  a  polygon  display  worse  than  an  integrated  bar  chart  display. 
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Coury  and  Pietras  (1989)  compared  the  following  three  display  formats:  graphic,  alphanumeric,  and  multiple 
formats.  Performance  in  a  simulated  process  control  task  was  worse  with  graphic  displays  and  best  with  the  multi- 
format  display.  Thus,  the  overall  results  on  the  effectiveness  of  polygon  displays  are  mixed. 

5.4.3  Conclusions 

In  general,  we  agree  with  the  conclusion  of  Bennett  and  Flach  (1992)  that  configural  displays  support  integrated 
task  performance,  and  that  a  mixed  pattern  of  results  is  found  when  focused  tasks  are  performed  (sometimes 
separable  displays  are  better  for  focused  tasks).  Further,  it  appears  that  configural  displays  can  be  enhanced  to 
better  support  focused  task  performance,  such  as  by  the  inclusion  of  digital  information.  However,  including 
digital  information  can  detract  from  the  emergent  features,  so  it  must  be  done  cautiously. 

The  aggregate  of  the  studies  related  to  information  representation  appears  to  support  the  PCP  that  performance  is 
best  supported  when  both  the  types  of  displays  and  their  details  closely  match  operators’  tasks.  Operators  of  NPPs 
have  a  wide  variety  of  tasks,  from  high-level  fault  detection  and  diagnosis  to  lower-level  component  manipulation. 
Thus,  one  can  conclude  that  operators  should  have  a  variety  of  different  levels  of  displays  that  span  the  A-A 
hierarchy.  (A  careful  review  of  NPP  instrumentation  and  displays  shows  that  to  some  extent  this  already  exists). 
The  operators  should  be  able  to  select  from  this  suite  of  displays  or  be  able  to  reconfigure  the  level  of  displays, 
based  on  the  task  at  hand.  When  this  level  of  diversity  of  displays  and  flexibility  in  selection  is  provided, 
performance  issues  naturally  arise  (see  Issue  [6]  below). 

In  general,  while  the  research  reported  in  this  section  was  more  rigorous  from  a  standpoint  of  experimental 
methodology  than  the  EID  studies,  these  studies  also  were  limited  in  generalizability,  because  of  the  following 
factors: 

•  Domain  representation  -  The  domains  represented  by  the  displays  did  not  address  the  complexity  of  real 
systems.  While  there  were  exceptions  (e.g.,  Woods  et  al.,  1982),  in  many  of  the  studies  the  domains 
represented  were  very  simple.  Since  the  emergent  features  are  meant  to  map  to  constraints  of  the  system, 
increasing  the  complexity  of  the  system  can  be  expected  to  make  the  mapping  considerably  more  complex. 

•  Tasks  -  In  part  due  to  simplifications  of  the  domain,  the  tasks  also  were  simple.  Even  the  fault  detection  in 
several  studies  was  simplified  and  based  on  only  a  few  variables.  Further,  the  studies  did  not  address  a  wide 
range  of  tasks. 

•  Participants  -  More  often  than  not,  the  participants  were  students  and  not  domain  experts.  Further,  they  did 
not  receive  the  training  that,  for  example,  Vicente’s  participants  did.  Thus,  issues  associated  with  the 
contribution  of  mental  models  to  interpreting  the  displays  did  not  reflect  the  complexity  of  interpretation  that 
would  arise  in  NPP  operations. 

The  limits  on  generalizability  give  rise  to  several  possible  research  issues,  as  noted  below.  Resolving  them  is 
necessary  for  developing  further  guidance. 

(1)  Lack  of  Specific  HFE  Guidance 

Bennett  and  Flach  (1992)  concluded  that  “Although  there  is  general  consensus  that  this  type  of  display  has  the 
potential  to  improve  human-machine  system  performance,  there  is  less  agreement  on  the  principles  or  heuristics 
that  should  be  used  as  design  guidelines”  (p.  514).  The  research  on  advanced  graphic  forms  is  at  an  early  stage, 
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and  while  we  can  formulate  a  few  general  guidelines,  the  research  is  not  yet  substantial  or  definitive  enough  to 
provide  a  solid  technical  basis  on  which  to  develop  a  complete  set  of  guidelines. 

(2)  System  Complexity  and  Emergent  Features 

As  the  number  of  vertices  in  a  polygon  display  increased,  the  displays  became  complex  and  performance  was 
reduced.  Further,  emergent  features  can  be  affected  by  unpredictable  interactions  between  component  parts  and 
produce  unintended  effects  that  may  be  misunderstood  (Hansen,  1995).  This  illustrates  the  issue  of  increasing  the 
complexity  of  the  underlying  domain  to  which  integral  and  configural  displays  map.  For  complex  dynamic 
systems,  such  a s  NPPs,  not  much  is  known  about  the  dynamic  aspects  of  the  emergent  features  that  may  be  used  to 
represent  them.  In  addition,  as  the  graphic  representation  increases  in  complexity,  so  does  the  display  grammar. 

(3)  Perceptual  Resolution 

Configural  and  integral  displays  require  a  perceptual  process  to  take  place,  such  as  recognizing  a  change  in  an 
emergent  feature.  However,  the  degree  to  which  a  geometric  form  needs  to  change,  before  it  is  perceived  as  a 
distortion,  is  not  well  understood. 

(4)  Elements  of  Configural  Display 

The  research  discussed  above  suggested  that  configural  displays  could  be  enhanced,  especially  in  support  of 
focused  tasks,  for  example,  by  including  digital  information.  Research  is  needed  to  better  understand  the  effects  of 
display  elements  on  performance  and  the  effects  of  their  interactions  with  other  types  of  displays. 

(5)  Evaluation  of  Display 

More  comprehensive  methods  of  evaluation  are  needed;  this  was  also  noted  in  Section  5.3  on  the  evaluation  of  EID 
displays.  For  example,  the  lack  of  definitive  review  guidance  will  need  to  be  compensated  for  with  dynamic 
evaluations.  The  criteria  for  the  evaluations  will  have  to  be  addressed.  Hansen  (1995)  suggested  a  signal  detection 
approach,  i.e.,  an  examination  of  hits,  false  alarms,  and  misses,  because  there  is  some  evidence  that  subjects  may 
generally  be  less  conservative  using  configural  displays  under  some  situations,  so  that  the  hits  as  well  as  the  false 
alarms  increase. 

(6)  Operator  Use  of  a  Large  Span  and  Variety  of  Displays 

If  and  when  the  operators  of  a  NPP  have  a  large  variety  of  different  levels  of  displays  from  which  to  choose,  how 
does  the  designer  decide  how  many  displays  are  enough?  Have  the  operators  been  given  too  much  information, 
either  in  a  single  display  or  in  the  entire  suite  of  displays  that  are  available?  Will  the  operators  be  able  to  select  the 
appropriate  display  for  the  task  or  tasks  at  hand?  Will  operators  tend  to  choose  a  few  “favorite”  displays  even 
though  they  may  not  be  the  most  appropriate  for  the  tasks?  If  operators  do  switch  displays  based  on  varied  tasks, 
will  they  pick  the  proper  display?  What  sort  of  training  should  be  developed  to  address  these  concerns? 

5.5  Organization  of  Displays 

The  investigation  of  display  representations  discussed  in  Section  5.4  addressed  predominantly  simple  systems, 
where  task  performance  was  accomplished  with  a  single  display.  Even  the  studies  directed  at  complex  systems 
such  as  NPPs  (discussed  in  Section  5.3)  looked  at  performance  with  a  single  display  page.  However,  real-world 
complex  systems  cannot  be  represented  with  a  single  graphic  display  (Bennett  et  al.,  1997),  and  this  is  certainly 
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true  of  NPPs,  where  display  pages  number  in  the  hundreds  and  thousands  (O’Hara  et  al.,  1996).  Thus,  in  complete 
information  systems,  representations  are  combined  into  display  pages  and  pages  into  networks. 

There  are  two  characteristics  of  computer-based  HSIs  that  impact  the  organization  of  information:  information 
volume  and  available  display  area.  Computer-based  HSIs  equipped  with  digital  I&C  systems  typically  provide 
much  more  real-time  information  than  conventional  CRs.  Gaddy  et  al.  (1991)  observed  “a  common  occurrence 
that  often  accompanies  the  introduction  of  computer-based  technologies  is  an  order-of-magnitude  increase  in  the 
quantity  of  information  at  the  user’s  fingertips”  (p.  256). 

An  interesting  paradox  is  that  while  the  volume  of  information  increases  considerably,  it  is  available  through  the 
very  limited  viewing  area  provided  by  workstation  VDUs.  This  characteristic  has  been  referred  to  as  the  “keyhole 
effect”  (Woods  et  al.,  1990),  since  at  any  given  time  most  of  the  information  is  hidden  from  view;  i.e.,  the  operator 
has  only  a  glimpse  of  the  current  plant  information  through  the  display  devices.  Therefore,  operators  must  know 
what  information  and  controls  are  available  in  the  virtual  information  space,  where  they  are,  and  how  to  navigate 
and  retrieve  them.  If  insufficient  viewing  area  is  available  for  operators  to  accomplish  their  tasks,  they  may  have 
to  frequently  repeat  navigation  tasks.  A  problem  related  to  the  keyhole  effect  is  that  access  to  controls  and  display 
tends  to  be  serial,  e.g.,  only  a  few  controls  can  be  accessed  at  one  time,  which  is  in  contrast  to  the  parallel 
presentation  of  controls  and  displays  in  conventional  CRs. 

Thus,  to  present  plant  information  in  a  limited  viewing  area,  multiple  levels  of  display  pages  are  needed,  and  each 
must  be  accessed  from  a  large  network  of  pages  before  it  is  viewed.  This  way  of  presenting  information  may 
interfere  with  the  ability  of  operators  to  quickly  scan  indications  and  assess  plant  conditions.  Not  all  desired 
information  may  be  presented  at  one  time  via  the  HSI.  Because  information  in  computer-based  CRs  must  be 
retrieved  from  display  networks,  operators  must  know  where  to  look  to  obtain  status  information.  That  operators 
may  only  monitor  what  they  can  observe  and  may  ignore  information  not  immediately  presented  has  been 
identified  as  a  human  performance  concern  associated  with  computer-based  systems  (Fujita,  1992;  Elm  and 
Woods,  1985;  Stubler  et  al.,  1991). 

Therefore,  decisions  on  the  design  of  display  pages  and  their  organization  into  networks  are  important 
considerations.  This  section  will  address  these  two  aspects  of  display  organization. 

5.5.1  Organization  of  Display  Pages 

The  limited  display  area  in  VDU-based  systems  means  that  when  information  is  needed  by  an  operator,  it  may  be 
contained  on  more  than  one  display  page,  which  may  require  the  operator  to  make  rapid  transitions  between  pages, 
try  to  remember  values,  or  write  values  on  paper.  Compared  to  the  sweeping  wall-to-wall  panels  of  older 
conventional  CRs,  CRs  using  advanced  information  systems  have  a  more  restricted  view,  even  when  multiple 
display  devices  are  provided.  Thus,  the  design  of  individual  pages  is  critical  to  performance.  The  design  review 
issues  related  to  display  pages  are  (1)  the  information  to  be  combined  into  a  single  display  page,  and  (2)  the 
interaction  of  different  display  formats  within  a  display  page.  However,  very  little  research  was  identified  that 
addressed  these  issues. 

Information  Requirements  for  Single  Pages 

In  Section  5.2,  the  cognitive  tasks  supported  by  the  information  system  were  discussed.  Those  tasks  included 
situation  assessment,  monitoring  and  detection  of  process  disturbances,  response  planning,  and  response 
implementation.  The  information  requirements  for  each  of  these  activities  are  different  and  a  task  analysis, 
perhaps  augmented  by  an  analysis  of  the  abstraction  hierarchy,  will  help  define  the  operator’s  information  needs 
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for  performing  these  cognitive  activities.  After  developing  the  overall  information  needs,  the  specific  organization 
and  coordination  of  the  information  into  distinct  pages  must  be  addressed. 

Design  considerations  for  display  pages  are  highly  related  to  and  dependent  on  other  factors,  notably  the  following: 

•  Design  of  information  (how  much  information  is  packed  into  individual  display  pages) 

•  Amount  of  display  area  that  is  available 

•  Interface  management  functions  (how  operators  move  within  the  organization  to  retrieve  information) 

These  characteristics  also  interact  with  each  other.  Since  the  amount  of  information  is  large  and  the  display  area 
limited,  more  information-dense  displays  lead  to  fewer  pages  and  less  need  to  manage  the  interface.  The  tradeoff 
is  that  very  dense  displays  can  reduce  the  salience  of  important  information  and  make  it  difficult  to  locate  needed 
information  within  the  display  page. 

Recent  research  showed  a  preference  for  a  high  density  of  information  in  displays.  Thomas  et  al.  (1998)  evaluated 
the  design  of  a  wall  panel  information  system  for  an  advanced  CR.  A  mimic-type  display  was  compared  with  a 
display  with  considerably  more  information  at  various  levels  of  abstraction.  They  found  the  crew’s  situation 
awareness  increased  and  workload  decreased  with  the  more  dense  design.  In  addition,  operators  preferred  the  new, 
more  dense  design.  In  this  study,  however,  operators  did  not  have  control  over  the  amount  of  information  on  the 
display. 

In  another  study  (Roth  and  O’Hara,  1998),  operators  had  control  over  the  amount  of  information  they  could  put  on 
the  display.  They  tended  to  cram  their  VDUs  with  information.  For  example,  they  would  place  multiple  trend 
plots  on  a  single  VDU.  It  was  not  unusual  for  operators  to  divide  a  VDU  screen  into  two  windows  and  place  four 
trend  plots  in  each.  Since  each  trend  plot  could  have  up  to  4  parameters  trended,  operators  could  have  up  to  32 
parameters  trended  per  VDU.  They  were  observed  putting  up  multiple  windows  per  VDU,  occasionally 
overlapping  and  covering  up  one  display  with  another.  Covering  up  information  by  overlapping  or  covering  one 
display  with  another  was  not  found  to  be  a  problem  in  this  study. 

An  important  reason  for  preferring  information -dense  displays  was  that  operators  rarely  modified  or  pulled  up  new 
displays  once  a  scenario  was  started.  Rather,  they  tended  to  select  a  set  of  trend  plots  to  display  at  the  beginning  of 
a  scenario  and  kept  that  arrangement.  This  finding  is  consistent  with  the  finding  that  operators  are  generally 
reluctant  to  engage  in  interface  management  tasks  in  high  workload  situations  (O’Hara  et  al.,  1997). 

Another  aspect  of  information  requirements  for  single  pages  is  how  complex  information  that  extends  beyond  the 
physical  limits  of  single  pages,  such  as  a  detailed  P&ID  or  procedure  flow  chart,  should  be  divided  to  form 
individual  pages.  Again,  tradeoffs  must  be  considered  with  approaches  for  interface  management. 

Representational  Format  Interactions 

While  operators  appear  to  prefer  information-dense  displays  (to  reduce  interface  management  tasks),  very  little 
research  has  been  conducted  on  the  effects  of  mixing  representation  formats.  Payne  and  Lang  (1995)  examined  the 
effects  of  mixed  vs.  pure  format  displays  on  performance  of  focused  vs.  separable  tasks.  In  pure  format  display,  all 
indicators  within  a  display  are  the  same,  either  words  (such  as  “fifty-five”),  digital  (such  as  “55”),  or  analog 
(graphic  representation).  In  mixed  format  displays,  a  variety  of  formats  are  presented.  They  hypothesized  that 
performance  on  focused  tasks  would  be  better  supported  with  mixed  formats.  Undergraduate  students  visually 
monitored  a  dynamic  simulated  system.  Their  task  was  to  indicate  whether  an  indicator  exceeded  the  safe  range. 
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Accuracy  and  latency  were  the  dependent  variables.  Responses  were  faster  with  the  analog  displays  than  with 
verbal  or  digital  displays.  There  was  no  support  for  the  contention  that  mixed  format  displays  would  better  support 
focused  tasks.  In  fact,  more  errors  were  committed  in  the  mixed  condition.  Thus,  mixing  formats  may  have 
undesired  effects  on  performance.  However,  the  results  of  this  study  are  suggestive  only.  The  displays, 
participants,  and  tasks  were  not  representative  of  a  NPP  context. 

5.5.2  Organization  of  Networks 

As  previously  noted,  NPPs  can  have  hundreds  or  even  thousands  of  display  pages.  Therefore,  their  organization 
into  a  network  is  essential  for  rapidly  and  efficiently  retrieving  information.  A  variety  of  schemes  for  organizing 
information  are  possible:  e.g.,  by  physical  equipment  layout,  systems,  functions,  and  operator  tasks.  While  there 
has  not  been  a  great  deal  of  research  on  the  organization  of  information  in  a  display  network,  numerous 
organizational  schemes  have  been  criticized.  Ends  ley  (1995)  criticizes  physical  organization,  and  Heslinga  and 
Herbert  (1993,  1995)  criticize  a  plant  systems  organization. 

The  most  common  organization  scheme  for  display  networks  in  NPPs  is  by  plant  functions  and  systems.  Such 
organizations  were  effective  for  the  layout  of  conventional  plants,  but  a  recent  survey  of  power  plants  suggests  it 
may  not  be  effective  for  organizing  display  networks.  Heslinga  and  Herbert  (1993,  1995)  surveyed  designers, 
managers,  and  operators  of  plants  of  various  types  from  eight  European  countries  on  their  experiences  with  the 
introduction  of  advanced  HSI  technology,  either  as  part  of  new  plant  design  or  plant  modernization.  They  found 
that  the  displays  differed  between  a  system-based  hierarchal  organization  (e.g.,  an  organization  around  plant 
systems  with  displays  arranged  hierarchically  from  overview  to  detailed  levels)  and  a  task-oriented  organization 
(e.g.,  an  organization  around  operator  tasks).  In  comparison  to  task-oriented  displays,  operators  reported  having 
problems  finding  information  in  system-based  organizations.  This  preference  was  especially  true  for  non-routine 
operations.  The  authors  suggested  that  operators’  information  needs  were  not  centered  along  the  system  hierarchy, 
but  along  task  requirements.  Since  the  system-based  organization  did  not  conveniently  provide  the  information 
needed  for  task  performance,  operators  had  to  search  through  multiple  displays.  Heslinga  and  Herbert  (1993) 
concluded  that  displays  “dedicated  to  a  certain  task  instead  of  the  currently  used  system-based  displays,  could  be  a 
valuable  alternative”  (p.  33).  For  non-routine  situations,  there  was  insufficient  time  to  search  for  information  and 
controls  within  a  hierarchy  of  displays,  leading  to  the  operators’  reluctance  to  search  through  functionally-oriented 
sets  of  display  pages.  However,  it  may  be  more  difficult  to  establish  task-based  displays  for  non-routine  situations, 
due  to  their  unpredictable  nature  and  potentially  large  number.  While  Heslinga  and  Herbert’s  observations  are 
important,  we  found  no  research  that  specifically  addressed  and  manipulated  the  organization  of  display  pages. 

Woods  and  Roth  (1988)  proposed  using  multiple  organizational  schemes.  They  discussed  fixed  vs.  adaptive 
collections  of  information.  Given  that  there  is  too  much  data  to  see  all  at  once,  the  question  is  how  can  the 
representational  design  be  manipulated  to  help  the  operator  find  the  right  set  at  the  right  time?  While  needed  data 
may  be  available  in  the  system,  the  data  needs  to  be  grouped  based  on  some  criterion,  such  as  where  the  device  is 
located,  physical  system  boundaries,  time,  or  function. 

When  the  designer  preselects  fixed  collections  of  data,  the  effect  on  performance  depends  on  the  amount  of 
searching  that  the  operator  must  do  across  the  predefined  groupings,  which  in  turn,  depends  on  (1)  the  degree  to 
which  the  designer’s  choice  of  criteria  anticipates  the  questions  to  be  asked  by  the  operator,  (2)  the  degree  to  which 
links  between  data  sets  and  domain  questions  vary,  and  (3)  the  characteristics  of  the  representation  in  which  the 
search  occurs  (e.g.,  parallel  vs.  serial  display). 

Since  the  questions  asked  may  vary,  adaptive  collections  of  data  may  be  best.  In  adaptive  collections,  the  grouping 
of  data  can  change  as  a  function  of  the  state  of  the  plant  or  of  the  operators’  intent.  Suppressing  alarms  based  on 
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plant  mode  is  an  example  of  grouping  changes  based  on  the  plant’s  state.  Operators,  however,  may  make  changes 
to  individual  display  pages,  create  new  ones,  or  even  globally  change  the  grouping  of  pages.  Such  global  changes 
could  be  predefined  (e.g.,  normal  operations,  startup,  normal  shutdown,  post-scram)  and  then  invoked  at  the 
operators’  discretion. 

5.5.3  Conclusions 

While  the  issues  of  organization  of  pages  and  networks  are  important  and  interact  with  many  other  design 
considerations,  little  data  were  found  on  the  organization  of  large,  complex  information  systems.  Some 
information  seemed  to  contradict  current  HFE  guidance  and  nuclear  practice.  That  is,  very  dense  displays  were 
found  to  lead  to  better  performance  and  to  be  preferred  by  operators.  The  common  industry  practice  of  network 
organization  by  function  and  system  was  found  to  inadequately  support  operators’  task  performance. 

Both  of  these  findings  may  reflect  the  effects  of  workload  associated  with  interface  management,  i.e.,  that  operators 
seek  to  minimize  the  demands  of  interface  management.  Less  dense  displays  increase  the  number  of  display  pages 
and  the  amount  of  interface  management  needed  to  retrieve  information.  Similarly,  organizing  display  pages  by 
plant  systems  increased  the  interface  management  workload  associated  with  performing  tasks.  A  further 
complicating  factor  may  be  the  flexibility  of  the  HSI,  which  may  inhibit  the  development  of  an  accurate  model  of 
the  display  structure.  If  operators  can  view  the  display  network  in  many  different  ways,  it  may  be  more  difficult  to 
understand  its  organization,  and  as  a  result,  navigation  plans  may  be  less  predictable.  Thus,  the  organization  of 
display  pages  and  of  networks  remain  human  performance  issues  and  further  research  to  address  them  is 
warranted. 

5.6  General  Summary  and  Issues 

The  purpose  of  Section  5  was  to  establish  a  technical  basis  upon  which  to  develop  review  guidance  for  information 
systems  and  to  identify  human  performance  issues  where  the  technical  basis  is  insufficient  for  developing 
guidance.  To  accomplish  this  objective,  we  examined  research  in  the  following  four  areas:  (1)  generic,  operator 
cognitive  tasks  that  an  information  system  must  support,  (2)  analysis  of  information  requirements,  (3) 
representation  of  information,  and  (4)  organization  of  information.  Section  5.6.1  summarizes  the  results  of  this 
research,  and  Section  5.6.2  gives  the  remaining  research  issues. 

5.6.1  Information  System  Design  and  Operator  Performance 

Cognitive  Tasks 


The  plant  information  systems  provide  the  basis  for  operators  to  perform  their  role  in  the  plant.  The  general  tasks 
that  the  information  systems  must  support  include  situation  assessment,  monitoring  and  detection,  response 
planning,  and  response  implementation.  The  following  general  observations  were  made: 

•  Operators’  information  needs,  especially  in  support  of  monitoring,  detection,  and  situation  assessment,  are 
very  broad,  both  in  terms  of  the  range  of  activities  that  operators  need  to  be  aware  of,  and  the  level  of 
abstraction  they  need  to  consider. 

•  Information  should  be  expressed  in  terms  that  are  at  the  level  that  will  directly  impact  decision  making  and 
control.  Cognitive  resources  should  not  be  required  to  mentally  process  data  to  obtain  information  that  is 
needed. 
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•  It  is  important  to  correctly  map  the  plant  or  process  and  the  identification  of  information  needs  and  their 
representation  to  the  operator’s  mental  model.  These  mappings  are  necessary  to  ensure  that  the 
representations  of  information  trigger  the  appropriate  mental  representation  of  the  situation. 

•  HSI  design  can  strongly  affect  the  operator’s  cognitive  activity  and  can  make  information  difficult  to  obtain 
(through  poor  organization,  high  data  density,  and  excessive  interface  management  demands). 

The  information  processing  basis  for  the  operators’  cognitive  tasks  should  be  considered  in  the  guidance  for 
reviewing  information  system  designs.  The  way  in  which  these  needs  are  reflected  in  the  information  system’s 
design  is  through  analysis  of  information  requirements,  representation,  and  organization. 

Analysis  of  Information  Requirements 

In  NPP  design,  a  systems  engineering  approach  was  recommended  to  identify  information  requirements.  In  such 
an  approach,  plant  functions  are  decomposed  based  on  performance  requirements  and  allocated  to  human  and 
machine  resources.  Human  functional  requirements  are  further  analyzed  in  task  analyses  to  better  define  the 
information  requirements  of  task  performance. 

Such  traditional  approaches  to  information  system  design  were  criticized  because  the  information  requirements 
they  define  could  fail  to  meet  the  operator’s  needs  in  the  face  of  unplanned  events.  EID  was  identified  as  a 
potential  alternative  approach.  There  are  four  aspects  of  the  EID  approach  (Section  5.3): 

•  Specification  of  information  requirements 

•  Organization  of  information  among  the  levels  of  an  abstraction  hierarchy 

•  Development  of  innovative  displays  to  show  functionality 

•  Addition  of  analytical  redundancy 


Each  may  have  some  performance-enhancing  features. 

In  specifying  information  requirements,  using  the  A-A  matrix  may  provide  a  framework  for  identifying 
information  needs  by  giving  a  structure  that  specifies  levels  of  functional  decomposition.  Further,  it  appears  that 
interfaces  based  on  EID  provide  better  support  for  knowledge-based  behavior  than  those  based  on  physical 
variables  alone.  By  using  EID  goals,  the  relevant  system  requirements,  constraints,  and  parameters  can  be 
evaluated  and  represented. 

Developing  innovative  displays  to  show  functionality  may  not  be  considered  purely  a  feature  of  EID,  but  once  the 
levels  of  functionality  that  should  be  displayed  are  specified,  the  designers  should  develop  appropriate  displays. 
Such  displays  will  probably  not  be  found  in  the  design  engineer’s  tool  box  of  past  successful  applications,  and  thus 
will  likely  need  to  be  developed  for  the  particular  application  in  question. 

The  addition  of  analytical  redundancy  is  another  feature  of  EIDs  that  is  not  always  applied,  but  which  appears  to  be 
quite  valuable.  This  is  a  particularly  effective  method  of  verifying  that  higher  level  functions  and  goals  are,  in  fact 
being  met,  and  if  they  are  not  being  met,  determining  why  and  what  to  do  about  it. 
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While  several  methodological  issues  were  raised  related  to  the  studies  of  EID  effects,  EID  designs  have  shown 
some  beneficial  effects.  These  effects  may  be  stronger  for  fault  diagnosis  and  disturbance  handling  than  for  normal 
operations,  and  these  aspects  may  be  important  to  safety.  Several  issues  remain  to  be  addressed;  they  are 
summarized  in  Section  5.6.2. 

Representation  of  Information 

Once  information  requirements  are  identified,  they  must  be  represented  in  the  information  system.  That  is,  display 
formats  are  developed  to  communicate  the  information  to  the  operators.  The  success  of  representational  aids  was 
discussed  as  being  a  function  of  the  characteristics  of  the  plant,  the  operators,  and  the  representation  through 
which  the  operators  view  and  act  on  the  plant.  All  three  are  essential  parts  of  the  overall  system.  The 
representation  involves  the  following  two  types  of  mapping: 

•  Correspondence  map  between  the  properties  and  characteristics  of  the  system  to  be  represented  and  the 
features  in  the  representation  (how  well  the  display  communicates  meaningful  information  about  the  plant  that 
is  needed  by  operators) 

•  Coherence  map  between  the  features  in  the  representation  and  the  physical  and  cognitive  characteristics  of  the 
operator  (how  comprehensible  the  representation  is  to  the  operator) 

Three  generic  types  of  displays  were  identified:  separable,  integral,  and  configural  displays.  Separable  displays 
are  representations  where  each  process  parameter  is  presented  individually  and  no  relationships  between  the 
parameters  are  provided  by  the  representation  itself.  Integral  displays  are  displays  which  show  the  integration  of 
information  in  such  a  way  that  the  individual  parameters  used  to  generate  the  display  are  not  represented  in  it. 
Configural  displays  provide  a  combination  of  the  other  two  types.  Information  about  the  basic  parameters  is 
available;  however,  information  on  their  relationships  becomes  evident  as  features  emerge  from  the  representation. 
An  emergent  feature  is  a  “high-level,  global  perceptual  feature”  that  is  produced  by  the  interactions  among 
individual  parts  or  graphical  elements  of  a  display. 

Research  studies  tended  to  compare  these  display  types  based  on  the  types  of  tasks  participants  performed.  Focused 
tasks  are  independent  from  each  other  in  terms  of  requiring  separate  information.  Integrated  tasks  required  the 
integration  of  information. 

In  general,  the  research  found  that  configural  displays  support  integrated  task  performance,  and  that  a  mixed 
pattern  of  results  is  found  when  focused  tasks  are  performed  (sometimes  separable  displays  are  better  for  focused 
tasks).  The  studies  found  that  performance  is  best  supported  when  the  representation  of  information  matches 
operator’s  tasks.  Operators  of  NPPs  have  a  wide  variety  of  tasks,  from  high-level  fault  detection  and  diagnosis  to 
lower-level  component  manipulation.  Thus,  one  can  conclude  that  operators  should  have  available  to  them  a 
variety  of  different  levels  of  displays  that  span  the  A-A  hierarchy. 

However,  most  of  the  studies  of  information  representation  also  were  found  to  be  limited  in  generalizability  due  not 
to  methodological  issues,  but  to  the  use  of  simple  systems,  tasks,  and  inexperienced  participants.  Thus,  the 
extension  of  the  research  findings  to  complex  systems  such  as  NPPs  must  be  made  with  caution. 

Organization  of  Information 


While  organization  of  display  pages  and  networks  is  important,  little  data  on  organization  of  large,  complex 
information  systems  were  found.  The  study  of  these  issues  is  complicated  due  to  the  interaction  of  the 
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information’s  organization  and  other  design  considerations,  such  as  display  area  available  and  interface 
management  functions. 

Some  of  the  research  seemed  to  contradict  current  HFE  guidance  and  nuclear  practice.  That  is,  very  dense  displays 
were  found  to  lead  to  better  performance  and  were  preferred  by  operators.  Additionally,  the  common  industry 
practice  of  organization  by  functions  and  systems  was  found  to  be  difficult  for  operators  to  use  in  some  studies. 

Both  of  these  findings  may  reflect  the  effects  of  workload  associated  with  interface  management,  i.e.,  operators 
seek  to  minimize  the  demands  of  interface  management.  Less  dense  displays  increase  the  number  of  display  pages 
and  the  amount  of  interface  management  needed  to  retrieve  information.  Similarly,  organizing  display  pages  for 
plant  systems  increased  the  workload  of  interface  management  associated  with  performing  tasks.  A  further 
complicating  factor  may  be  flexibility  of  the  HSI,  which  may  inhibit  the  development  of  an  accurate  model  of  the 
display’s  structure.  If  operators  can  view  the  display  network  in  many  different  ways,  it  may  be  more  difficult  to 
understand  its  organization,  and  as  a  result,  navigation  plans  may  be  less  predictable.  Thus,  the  organization  of 
display  pages  and  of  networks  are  human  performance  issues,  and  further  research  is  needed  to  address  them 

5.6.2  Issues 

This  section  summarizes  the  human  performance  issues  associated  with  information  systems  that  were  identified 
based  on  the  literature  review.  The  issues,  as  described  below,  are  not  mutually  exclusive.  They  overlap,  and  some 
are  more  general  than  others;  thus,  some  may  be  considered  secondary  to  others.  As  the  issues  all  pertain  to  the 
interactions  within  an  integrated  human-machine  system,  interdependencies  are  unavoidable.  The  issues  are 
organized  into  the  following  three  sections  below: 

•  Technical  Basis  Issues 

•  Design  Review  Issues 

•  Operator-Related  Issues 

From  a  researcher’s  standpoint,  these  issues  represent  topics  for  which  research  is  necessary  before  additional 
guidance  can  be  developed.  From  a  regulatory  review  perspective,  many  of  them  can  be  addressed  on  a  case-by- 
case  basis  as  part  of  the  design  process  review  discussed  in  Part  2  of  this  document. 

5.6.2.1  Technical  Basis  Issues 

There  were  five  issues  identified  in  the  Technical  Basis  area. 

(1)  Lack  of  a  Well-Defined  EID  Process 

While  the  EID  approach  may  be  a  promising  advance  for  the  system  engineering  process,  developing  a  well- 
defined  process  for  conducting  an  analysis  using  the  abstraction  hierarchy  is  important  to  its  broader  application  to 
the  design  process. 
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(2)  Lack  of  Specific  Representation  Guidance 

The  research  on  advanced  graphic  forms  is  still  at  an  early  stage,  and  while  a  few  general  guidelines  can  be 
established,  the  research  is  not  yet  substantial  or  definitive  enough  to  provide  a  solid  technical  basis  on  which  to 
develop  a  complete  set  of  guidelines. 

(3)  Evaluation  of  Operating  Experience 

For  those  EID  displays  which  were  installed  in  operating  facilities,  the  operating  experience  and  its  application  to 
EID  should  be  thoroughly  assessed.  Such  experience  is  important  to  formulating  criteria  for  design  review  to 
address  EID  aspects  of  the  HSI. 

(4)  Critical  Testing  and  Evaluation  of  EID  Concepts 

There  has  not  been  much  thorough  research  that  carefully  assesses  the  various  aspects  of  EID,  e.g.,  information 
requirements,  effect  of  organization  of  information  along  functional  lines,  display  representation,  and  use  of 
analytical  redundancy.  Studies  tend  to  confound  these  characteristics  or  provide  weak  assessments  of  the 
contribution  of  the  various  aspects  of  EID  concepts. 

The  successful  handling  of  unplanned  or  unanticipated  events  with  EID  displays  under  actual  operational 
conditions  in  complex  systems  has  not  been  demonstrated. 

Further  research  also  is  necessary  to  more  clearly  determine  which  ceils  of  the  A-A  matrix  are  important  to 
operations. 

(5)  Evaluation  of  Displays 

More  comprehensive  methods  for  evaluating  displays  are  needed.  For  example,  the  lack  of  definitive  review 
guidance  will  need  to  be  compensated  for  with  dynamic  evaluations.  The  criteria  for  the  evaluations  will  have  to 
be  addressed.  For  example,  signal  detection  approaches  (i.e.,  an  examination  of  hits,  false  alarms,  and  misses) 
were  recommended.  This  is  due  to  some  evidence  that  subjects  may  generally  be  less  conservative  using  configural 
displays  under  certain  situations,  so  that  the  hits  as  well  as  the  false  alarms  increase  (Hansen,  1995). 

S.6.2.2  Design  Review  Issues 

There  were  ten  issues  identified  in  the  area  of  Design  Review. 

(1)  Task  and  Temporal  Considerations 

The  A-A  matrix  and  the  EID  approach  address  the  plant  at  a  functional  level  rather  than  on  a  task  or  temporal 
basis.  However,  the  importance  of  presenting  information  consistent  with  task  requirements  has  been  a 
fundamental  basis  of  the  systems  approach,  and  deviating  from  it  is  a  problem  in  many  new  plant  information 
systems  (where  information  is  organized  around  plant  systems  rather  than  operator  tasks). 

Much  of  the  operators’  goal-directed  activity  is  centered  around  temporal  constraints,  which  is  one  reason 
operators  like  trend  displays.  The  operators’  tasks  unfold  along  a  temporal  continuum  with  some  tasks  being 
performed  in  parallel  and  some  sequentially.  This  temporal  dimension  to  operations  may  be  lacking  from  the  A-A 
analysis,  except  where  time  is  reflected  in  higher-level  information,  such  as  rates. 
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Additionally,  the  EID  approach  seems  to  be  built  around  functional  decomposition  of  means-ends  relationships. 
How  well  will  this  relate  to  common  tasks  and  disturbances,  that  are  highly  proceduralized,  such  as  in  EOPs? 

While  it  is  important  to  be  able  to  address  unanticipated  and  unplanned  events,  still  most  of  the  operator’s 
activities  are  well  planned  and  the  information  system  needs  to  support  these  activities  as  well.’ 

The  proper  role  of  task-based  information  in  the  design  of  displays  and  how  it  is  integrated  into  the  EID  approach 
needs  to  be  addressed. 

(2)  Volume  of  Information 

Due  to  the  extensive  analytical  process  used  in  EID,  there  is  the  potential  that  the  process,  including  the  A-A 
matrix,  may  lead  to  identifying  more  information  than  can  be  displayed  practically.  Also,  too  many  display  pages 
may  be  required  to  satisfy  information  requirements  identified  by  EID. 

(3)  Density  of  Display  Information 

The  increase  in  information  may  be  linked  to  an  increase  in  the  density  of  information  on  individual  displays. 
While  this  may  be  done  to  minimize  the  need  for  interface  management  tasks,  such  as  navigating  to  retrieve 
additional  display  pages,  the  density  may  be  associated  with  a  lack  of  salience  of  important  information.  Another 
issue  with  dense  displays  is  that,  for  any  given  operator  task,  the  amount  of  irrelevant  information  increases,  and 
from  a  human  performance  perspective,  performance  decreases  as  the  amount  of  irrelevant  data  increases  (Mitchell 
and  Miller,  1983). 

(4)  Operator  Use  of  a  Large  Span  and  Variety  of  Displays 

If  and  when  the  operators  of  a  NPP  have  a  large  variety  of  different  levels  of  displays  from  which  to  choose,  how 
does  the  designer  decide  how  many  displays  are  enough?  Have  the  operators  been  given  too  much  information, 
either  in  a  single  display  or  in  the  entire  suite  of  displays  that  is  available?  Will  the  operators  be  able  to  select  the 
appropriate  display  for  the  task  or  tasks  at  hand?  Will  operators  tend  to  choose  a  few  “favorite”  displays  even 
though  they  may  not  be  the  most  appropriate  for  the  tasks?  If  operators  do  switch  displays  based  on  varied  tasks, 
will  they  pick  the  proper  one?  What  sort  of  training  should  be  developed  to  address  these  concerns? 

(5)  System  Complexity  and  Emergent  Features 

As  the  number  of  vertices  in  a  polygon  display  increased,  the  displays  became  complex  and  performance  was 
reduced.  Further,  emergent  features  can  be  affected  by  unpredictable  interactions  between  component  parts  and 
produce  unintended  effects  that  may  be  misunderstood  (Hansen,  1995).  This  illustrates  the  issue  of  increasing  the 
complexity  of  the  underlying  domain  to  which  integral  and  configural  displays  map.  For  complex  dynamic 
systems,  such  as  NPPs,  not  much  is  known  about  the  dynamic  aspects  of  the  emergent  features  that  may  be  used  to 
represent  them.  In  addition,  as  the  graphic  representation  increases  in  complexity,  so  does  the  display’s  grammar. 

(6)  Perceptual  Resolution 

Configural  and  integral  displays  require  a  perceptual  process  to  take  place,  such  as  recognizing  a  change  in  an 
emergent  feature.  However,  the  degree  to  which  a  geometric  form  needs  to  change  before  it  is  perceived  as  a 
distortion  is  not  well  understood. 
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(7)  Elements  of  Configural  Display 

The  research  suggested  that  configural  displays  could  be  enhanced,  especially  in  support  of  focused  tasks,  for 
example,  by  including  digital  information.  Research  is  needed  to  better  understand  the  effects  of  display  elements 
on  performance  and  the  effects  of  their  interactions  with  other  types  of  displays. 

(8)  Effect  of  Instrumentation  Failures 

The  effects  of  instrumentation  failures  on  EID  displays  is  a  potentially  significant  problem  that  needs  to  be 
addressed.  There  are  several  subissues  associated  with  this  problem. 

•  Can  operators  detect  a  failure  of  instrumentation? 

•  Can  failures  of  instruments  result  in  representations  that  are  interpreted  by  operators  as  real  process  failures; 
and,  perhaps  more  importantly,  can  real  process  failures  be  misinterpreted  as  failures  of  instruments? 

•  If  operators  detect  a  failure,  should  they  suspend  use  of  the  display? 

•  Since  the  display  integrates  many  parameters  into  a  single  display,  what  is  the  effect  of  its  loss  on  operations 
and  how  effectively  can  operators  switch  to  backup  displays? 

Many  designers  have  implemented  advanced  features  that  address  this  concern.  For  example,  the  concepts  of 
redundancy  and  diversity  in  instrumentation,  coupled  with  automatic  parameter  validation  and  appropriate 
notification  of  operators  when  instruments  fail  their  validation,  were  demonstrated  successfully. 

(9)  Organization  of  Information 

The  issues  related  to  organization  of  display  pages  and  networks  remain  as  important  research  topics. 

(10)  Integration  of  EID  Displays  into  Remainder  of  HSI 

The  integration  of  a  new  and  significantly  different  EID  display  into  the  remainder  of  the  standard  HSI  of  a  CR  is 
an  important  consideration  that  needs  to  be  addressed. 

S.6.2.3  Operator-Related  Issues 

There  were  three  Operator-Related  issues. 

(1)  Implications  for  Training  and  Qualification 

Christoffersen  et  al.  (1995)  concluded  that  “...to  experience  the  benefits  of  EID,  it  seems  likely  that  operators  need 
to  be  trained  to  think  functionally  rather  than  procedurally.  It  would  seem  that  this  would  require  a  fundamental 
shift  in  NPP  operation  philosophy.. .it  may  be  that  operators  have  to  possess  certain  types  of  cognitive 
characteristics  that  may  not  be  considered  in  the  traditional  selection  process  in  the  nuclear  industry”  (p.  143). 
Concern  was  also  expressed  that  EID  interfaces  may  inhibit  long-term  learning  and  retention  (Wickens,  1992). 

Also,  for  any  display  where  the  information  is  sophisticated  and  a  large  amount  is  consolidated  into  one  or  a  few 
figures,  the  requirements  for  training  become  very  important.  One  needs  to  understand  all  of  the  aspects  of  the 
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display  and  how  it  reacts  to  various  operational  transients,  accidents,  and  instrument  failures.  The  long-term 
effects  of  EID-type  displays  on  operators’  performance  and  strategies  are  unknown. 

The  training  aspects  of  Design  Review  Issue  (4),  Operator  Use  of  a  Large  Span  and  Variety  of  Displays,  should 
also  be  addressed. 

(2)  Acceptance  by  Operators 

The  issue  of  operators’  acceptance  of  a  new  and  different  type  of  display  is  also  important,  as  shown  by  their 
comments  during  experiments.  One  suggestion  from  the  Rankine  cycle  experiments  was  to  introduce  the  displays 
initially  in  training  and  then  perhaps  gradually  into  the  plant. 

(3)  Internal  vs.  External  Mental  Models 

Section  5.3  discussed  the  issue  of  the  appropriate  model(s)  to  apply  as  a  basis  for  designing  the  display.  Providing 
a  model  that  accurately  characterizes  the  process  and  its  supporting  systems,  yet  that  reflects  operators’  training, 
experience,  and  cognitive  capabilities,  is  an  important  factor.  Designing  displays  that  characterize  the  system  in 
ways  that  may  not  reflect  the  cognitive  requirements  of  plant  operators  to  perform  situation  assessment,  monitoring 
and  detection,  response  planning  and  response  implementation,  may  result  in  negative  performance.  Achieving 
this  balance  will  require  additional  research. 
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NUREG-07 1 1  provides  the  high-level  design  process  criteria  used  by  the  NRC  for  reviewing  the  overall  HFE 
programmatic  goals  and  objectives.  NUREG-07 1 1  does  not  address  the  detailed  review  of  final  HSI  designs  as 
implemented,  such  as  displays,  control,  or  procedures;  rather,  it  references  NUREG-0700  for  detailed  guidance.  In 
Part  1  of  NUREG-0700,  Rev.  1,  the  design  process  is  covered.  The  general  framework  of  NUREG-07 1 1  was  used 
to  structure  the  design  review  process  developed  in  NUREG-0700.  However,  Part  1  also  considers  aspects  of  the 
design  process  of  the  HSI  review  in  general  terms;  i.e.,  it  does  not  identify  the  specific  considerations  that  may  be 
important  in  reviewing  an  individual  technology,  such  as  information  systems.  The  only  detailed  HSI  technology- 
specific  guidelines  are  in  Part  2  of  NUREG-0700.  They  relate  to  the  detailed  form  and  functional  characteristics  of 
HSIs,  such  as  displays  and  controls,  and  not  the  unique  design  process  considerations. 

For  information  systems,  both  types  of  guidance  are  necessary  to  perform  a  design  review.  That  is,  while  there  is  a 
sufficient  technical  basis  to  develop  detailed  guidance  for  design  implementation  for  some  characteristics  of 
information  systems,  as  is  typical  in  NUREG-0700,  several  limitations  in  the  technical  basis  were  identified  (see 
Section  5.6.2. 1).  Thus,  there  are  many  issues  remaining  for  which  typical  NUREG-0700  guidance  could  not  be 
developed.  Until  it  can,  these  issues  can  be  addressed  for  specific  information  systems.  To  support  the  latter, 
considerations  were  identified  for  reviewing  the  information  system  design  process.  Thus,  guidance  was  developed 
to  address  both  the  design  process  and  human  factors  engineering  design.  Each  is  described  below. 

6.1  Guidance  for  Reviewing  the  Information  System  Design  Process 

Guidelines  were  developed  to  address  important  design  process  considerations  identified  in  the  literature,  and  to 
provide  a  place  where  human  performance  issues  could  be  identified  and  assessed,  case-by-case,  during  a  design 
review.  The  format  of  the  guidelines  corresponds  to  the  NRC’s  general  design  process  guidance  NUREG-07 1 1 . 
They  were  organized  into  the  following  sections: 

•  Operating  Experience  Review 

•  Function  and  Task  Analysis 

•  Human-System  Interface  Design 

•  Training  Program  Development 
These  guidelines  are  contained  in  Section  9. 

6.2  Guidelines  for  HFE  Design  Review 

A  draft  set  of  guidelines  was  developed  from  the  findings  and  source  materials  discussed  in  Section  5.  The 
methodology  was  conservative  in  the  sense  that  guidelines  were  developed  only  for  those  aspects  of  display  design 
that,  in  our  interpretation,  are  supported  by  the  literature.  Many  of  the  research  studies  evaluated  were  either  weak 
in  experimental  methodology  or  limited  in  generalizability  to  complex  systems.  This  situation  constrains  the 
development  of  new  HFE  guidelines. 

The  guidelines  adopted  the  standard  format  in  NUREG-0700,  Rev.  1 .  An  example  is  presented  below: 
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10.1-1  Correspondence  Mapping 

There  should  be  an  explicit  mapping  between  the  characteristics  and  functions  of  the  system  to  be  represented 
and  the  features  of  the  display  representation,  i.e.,  changes  in  the  appearance  of  the  display  form  should  have  a 
one-to-one  relation  with  the  plant  states  it  represents.  These  changes  should  result  from  explicit  rules  relating 
the  physical  form  of  the  display  and  its  meaning  with  respect  to  the  plant  state  represented. 

ADDITIONAL  INFORMATION:  Correspondence  mapping  addresses  how  well  the  display  communicates 
meaningful  information  about  the  plant  to  operators.  The  physical  form  and  functions  of  the  display  should  be 
explicitly  tied  to  its  meaning  with  respect  to  the  plant’s  functions  and  states.  The  meaning  of  the  display  must 
consider  the  instrumentation  and  the  data  processing  that  drives  the  display  format  If  a  single  display  can 
lead  to  more  than  one  interpretation,  the  display  is  ambiguous  and  can  be  more  easily  misunderstood. 

Changes  in  the  graphic  display  should  be  unambiguously  related  to  the  plant’s  state.  The  same  graphic 
change  should  not  be  associated  with  more  than  one  interpretation. 

Discussion :  Woods  and  Roth  (1988)  conceptualized  displays  as  involving  a  cognitive-system  triad  consisting 
of  the  world  to  be  acted  on,  the  agents  who  will  act  on  the  world,  and  the  representation  through  which  the 
agents  act  on  the  world.  The  success  of  a  representation  is  a  function  of  two  types  of  mapping  (Bennett  et  al., 
1997):  correspondence  and  coherence.  (Coherence  is  addressed  in  Guideline  10.1.3.)  The  importance  of 
providing  consistent  one-to-one  mapping  between  the  work  domain’s  constraints  and  the  features  of  the 
display  was  identified  as  an  important  principle  derived  from  research  on  representing  complex  systems 
(Bennett,  Nagy,  and  Flach,  1997;  Vicente  and  Rasmussen,  1992). 

Each  of  the  guidelines  is  composed  of  the  following  components: 

Guideline  Number -Within  each  section,  individual  guidelines  are  numbered  consecutively.  Each  guideline  has  a 
number  which  reflects  its  section  and  subsection  location,  followed  by  a  dash,  and  then  its  unique  number. 

Guideline  Title  -  Each  guideline  has  a  brief,  unique,  descriptive  title. 

Review  Criterion  -  Each  guideline  contains  a  statement  of  an  HSI  characteristic  so  that  the  reviewer  may  judge  the 
HSI’s  acceptability.  The  criterion  is  not  a  requirement,  and  discrepant  characteristics  may  be  judged  acceptable 
based  on  the  procedures  in  the  review  process. 

Additional  Information  -  For  many  guidelines,  there  is  additional  information  which  may  address  clarifications, 
examples,  exceptions,  and  details  on  measurements,  figures,  or  tables.  This  information  is  intended  to  support  the 
reviewer’s  interpretation  or  application  of  the  guideline. 

Discussion  -  This  section  summarizes  the  technical  basis  on  which  the  guideline  was  developed.  It  may  identify 
the  primary  source  documents,  the  technical  literature  such  as  journal  articles,  or  the  general  principle  from  which 
the  guideline  was  derived.  This  section  will  be  removed  when  the  guidance  is  integrated  into  NUREG-0700. 

In  place  of  the  Discussion  section  will  be  a  Source  field. 

Source  -  The  source  field  identifies  the  NUREG  or  NUREG/CR  (or  other  document)  containing  the  technical  basis 
and  development  methodology  for  the  guideline.  As  is  the  standard  practice,  the  source  field  will  cite  this 
document  as  it  will  appear  in  its  final  form. 

The  guidelines  were  organized  into  the  following  sections: 

•  Guidelines  for  General  Display 
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•  Display  Format 

-  Integral  Formats 

-  Configural  Display  Formats 

The  guidance  was  submitted  for  peer  review  and  was  revised  accordingly. 

The  new  review  guidelines  for  information  system  design  will  be  integrated  with  the  information  system  guidance 
in  NUREG-0700,  Rev.  1. 
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The  objective  of  this  study  was  to  develop  HFE  review  guidance  for  information  systems  based  on  a  technically 
valid  methodology  for  guidance  development.  To  support  this  objective,  the  following  tasks  were  performed: 

•  Development  of  a  framework  for  describing  key  design  characteristics  of  information  systems 

•  Development  of  a  technical  basis  using  human  performance  research  and  analyses  pertinent  to  information 
systems  and  related  areas 

•  Development  of  HFE  review  guidelines  for  information  systems  in  a  format  consistent  with  NUREG-0700, 
Rev.  1 ,  and  other  NRC  review  guidance 

•  Identification  of  remaining  issues  regarding  information  systems  for  which  research  was  insufficient  to  support 
the  development  of  NRC  review  guidance 

The  status  of  each  will  be  briefly  addressed  below. 

Characterization  Framework  for  Information  Systems 

The  first  step  in  guidance  development  was  to  identify  the  areas  for  which  guidance  was  needed.  Existing 
information  systems  were  reviewed  to  identify  the  dimensions  and  characteristics  along  which  such  systems  can  be 
defined.  Characterization  was  important  because  it  provided  a  structure  within  which  the  reviewer  could  request 
information  about  a  system,  and  with  which  to  structure  the  guidance. 

The  characterization  of  information  systems  included  the  following  major  components:  information  requirements, 
representation  system,  management  functions,  and  display  devices.  Representation  systems  were  further 
hierarchically  broken  down  into  display  elements  (basic  building  blocks  of  displays,  such  as  axes,  alphanumeric 
elements,  and  icons),  display  format  (such  as  P&ID  and  trend  graphs),  display  pages  (a  defined  set  of  information 
that  is  intended  to  be  displayed  as  a  single  unit),  and  display  networks  (the  organization  of  display  pages).  This 
report  mainly  addressed  information  requirements  and  representation. 

i 

Technical  Basis  Development 

The  development  of  detailed  review  guidelines  began  with  the  collection  of  technical  information  on  which  to  base 
the  guidance.  Research  in  the  following  areas  was  examined:  (1)  generic  cognitive  tasks  that  an  information 
system  must  support,  (2)  analyses  of  information  requirements,  (3)  representation  of  information,  and 
(4)  organization  of  information. 

The  plant’s  information  systems  are  the  basis  for  operators  to  perform  their  roles.  The  general  tasks  that  the 
information  system  must  support  include  situation  assessment,  monitoring  and  detection,  response  planning,  and 
response  implementation. 

The  information  processing  requirements  of  the  operators’  cognitive  tasks  have  to  be  reflected  in  the  design  review 
guidance  for  the  information  system.  The  way  in  which  these  needs  are  reflected  in  designing  information  systems 
is  in  analyses  of  information  requirements,  the  representation  of  information,  and  the  organization  of  information. 

In  NPP  design,  a  systems  engineering  approach  has  been  recommended  for  identifying  information  requirements. 
Plant  functions  are  broken  down  based  on  performance  requirements  and  allocated  to  humans  and  machines. 
Human  functional  requirements  are  further  analyzed  to  better  define  the  information  required  to  perform  the  tasks. 


7-1 


NUREG/CR-6633 


7  GENERAL  SUMMARY 


EID  was  identified  as  a  potential  additional  approach.  Its  four  aspects,  discussed  to  varying  extents  in  the  studies 
reviewed  in  Section  5.3,  are  the  following: 

•  Specification  of  information  requirements 

•  Organization  of  information  among  the  levels  of  an  abstraction  hierarchy 

•  Development  of  innovative  displays  to  show  functionality 

•  Addition  of  analytical  redundancy 

Each  of  them  may  have  some  performance-enhancing  features.  However,  the  associated  research  had 
methodological  weaknesses  that  limited  the  conclusions  that  could  be  drawn. 

Once  information  requirements  are  identified,  they  must  be  represented  in  the  information  system;  that  is,  display 
formats  are  developed  to  communicate  the  information  to  the  operators.  The  success  of  representational  aids  was 
discussed  as  being  a  function  of  the  characteristics  of  the  plant,  the  operators,  and  the  representation  through 
which  the  operators  view  and  act  on  the  plant.  All  three  are  essential  parts  of  the  overall  system. 

A  variety  of  approaches  to  representation  were  discussed,  including  separable  displays  (individual  parameters 
displayed  in  unrelated  form),  integral  displays  (individual  parameters  integrated  into  object  display  where  the 
individual  ones  are  not  visible),  and  configural  displays  (individual  parameters  are  displayed,  but  emergent 
perceptual  features  are  created  by  the  arrangement  and  organization  of  parameters). 

In  general,  some  research  support  was  found  for  the  representation  principles;  however,  most  of  the  studies  could 
not  be  generalized  due  to  the  use  of  simple  systems  and  tasks,  and  inexperienced  participants.  Thus,  extending  the 
research  findings  to  complex  systems  such  as  NPPs  must  be  done  with  caution. 

While  the  organization  of  the  display  page  and  network  are  important,  there  were  few  data  on  organizing  large, 
complex  information  systems.  Such  studies  are  complicated  due  to  the  interactions  between  informational 
organization  and  other  design  considerations,  such  as  the  display  area  available  and  interface  management 
functions.  Further,  some  research  seemed  to  contradict  current  HFE  guidance  and  nuclear  practice;  that  is,  very 
dense  displays  were  found  to  lead  to  better  performance  and  were  preferred  by  operators,  and  the  industry’s 
common  practice  of  function-system  network  organization  was  found  to  be  difficult  for  operators  to  use  when 
performing  certain  tasks. 

HFE  Review  Guidelines 


Once  the  technical  information  was  assembled,  a  draft  set  of  guidelines  was  developed.  The  guidelines  were 
organized  and  specified  in  a  standard  format  and  are  given  in  Part  2  of  this  document.  In  general,  guidelines  were 
only  developed  for  those  aspects  of  display  design  that,  in  our  interpretation,  are  supported  by  the  literature.  As 
discussed  in  Section  5,  many  research  studies  were  either  weak  in  experimental  methodology  or  limited  in 
generalizability  to  complex  systems.  This  situation  constrains  the  development  of  new  HFE  guidelines. 

Guidance  was  developed  to  support  (1)  Design  Process  Review,  and  (2)  HFE  Design  Review,  both  of  which  are 
needed  to  evaluate  an  information  system  design.  While  there  was  a  sufficient  technical  basis  to  develop  detailed 
design-implementation  guidance,  as  is  typical  in  NUREG-0700,  several  limitations  in  the  technical  basis  were 
identified,  and  so,  many  issues  remain  for  which  typical  NUREG-0700  guidance  could  not  be  developed.  However, 
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until  it  can  be  developed,  these  issues  should  be  addressed  for  specific  information  systems  case-by-case.  To 
support  reviewing  such  issues,  design  process  review  guidance  was  developed. 

The  new  guidance  will  be  integrated  into  the  existing  guidance  on  information  systems  in  NUREG-0700,  Rev.  1 . 
Information  Systems  Issues 

Where  there  was  insufficient  information  to  provide  a  technical  basis  upon  which  to  develop  valid  design  review 
guidance,  an  issue  was  defined.  There  were  several  human  performance  issues  associated  with  information 
systems  (in  Section  5.6.3).  The  issues  were  organized  into  three  topic  areas: 

•  Technical  Basis  Issues 


-  Lack  of  a  Well-Defined  EID  Process 

-  Lack  of  Specific  Representation  Guidance 

-  Evaluation  of  Operating  Experience 

-  Critical  Testing  and  Evaluation  of  EID  Concepts 

-  Evaluation  of  Displays 

•  Design  Review  Issues 

-  Task  and  Temporal  Considerations 

-  Volume  of  Information 

-  Density  of  Display  Information 

-  Operator  Use  of  a  Large  Span  and  Variety  of  Displays 

-  System  Complexity  and  Emergent  Features 

-  Perceptual  Resolution 

-  Elements  of  Configural  Display 

-  Effect  of  Instrumentation  Failures 

-  Organization  of  Information 

-  Integration  of  EID  Displays  into  the  Remainder  of  the  HSI 

•  Operator-Related  Issues 

-  Implications  for  Training  and  Qualification 

-  Acceptance  by  Operators 

-  Internal  vs.  External  Mental  Models 
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9  REVIEW  GUIDANCE  FOR  THE  INFORMATION  SYSTEM 

DESIGN  PROCESS 


Guidance  for  reviewing  the  design  process  for  information  systems  was  developed  to  address  important  design 
process  considerations  identified  in  the  literature  and  to  provide  a  means  whereby  human  performance  issues  may 
be  assessed  during  a  design  review  (see  Section  6.1).  The  review  guidelines  were  formatted  to  correspond  to  the 
NRC’s  general  design  process  guidance  in  NUREG-071 1 .  The  guidelines  are  organized  into  the  following 
sections: 

•  Operating  Experience  Review 

•  Function  Analysis 

•  Task  Analysis 

•  Human-System  Interface  Design 

•  Training  Program  Development. 

Guidelines  may  specify  that  some  aspect  of  an  information  system  needs  to  be  “evaluated.”  General  approaches  to 
evaluation  methods  and  criteria  development  are  defined  in  NUREG-0700.  Since  the  guidance  in  this  section  will 
eventually  be  incorporated  into  that  document,  the  methods  by  which  such  evaluation  should  be  performed  and  by 
which  criteria  should  be  identified  are  not  repeated  below. 

9.1  Operating  Experience  Review  (OER) 

(1)  Available  operating  experience  with  advanced  information  systems  should  be  reviewed  to  take  advantage  of 
lessons  learned  in  the  operational  use  of  the  systems,  as  well  as  to  ensure  that  problematic  aspects  of  their 
design  implementation  are  addressed. 

Discussion :  There  has  not  been  much  operating  experience  with  advanced  graphic  displays  in  NPPs.  An 
analysis  of  the  available  experience  will  help  ensure  the  design  supports  the  intended  aspects  of  operational 
performance.  This  is  consistent  with  the  NUREG-071 1  criterion  to  review  the  operating  experience  associated 
with  approaches  for  implementing  HSI  designs. 

9.2  Function  and  Task  Analysis 

(1)  The  function  analysis  and  task  analysis  criteria  by  which  information  requirements  are  defined  should  be 
clearly  documented. 

Discussion:  In  developing  a  higher-level  display,  such  as  one  based  on  EID  principles,  a  clear  definition  is 
needed  of  what  higher-level  functions  and  goals  are  addressed,  e.g.,  electric  production  and  safety.  For  the 
latter  goal,  the  NRC  established  five  critical  safety  functions  needed  in  NPPs  to  achieve  overall  safety: 
reactivity  control,  reactor  core  cooling  and  heat  removal,  RCS  integrity,  radioactivity  control,  and  containment 
conditions  (NUREG-1342  -  NRC,  1988).  These  should  be  addressed  in  high-level  safety  displays;  Tables  2 
and  3  of  NUREG-1342  give  sample  parameters  for  both  BWRs  and  PWRs. 

(2)  The  information  requirements  established  in  advanced  graphical  displays  should  be  based  on  the  operators’ 
cognitive  tasks  (i.e.,  situation  assessment,  monitoring  and  detection,  response  planning  and  response 
implementation). 
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Discussion.  Advanced  graphical  displays  are  integral  and  configural  displays  that  use  novel  graphic  forms  to 
convey  plant  information,  such  as  the  Rankine  cycle  display.  Information  defined  by  requirements  analyses, 
such  as  task  analysis  and  abstraction-hierarchy  analysis,  should  consider  the  operator’s  use  of  the  information 
and  the  specific  cognitive  tasks  it  will  be  used  for.  This  will  help  ensure  that  the  information  is  specified  at  the 
appropriate  level  for  the  operators’  needs. 

(3)  The  analysis  of  information  requirements  should  consider  the  different  needs  of  individual  staff  in  the  control. 
Discussion :  Roth  and  O’Hara  (1998)  identified  the  different  information  needs  of  the  shift  supervisor,  reactor 
operator,  balance-of-plant  operator,  and  shift  technical  advisor.  They  vary  both  in  terms  of  specific 
information  requirements,  as  well  as  in  the  level  of  abstraction  needed. 

9.3  Human-System  Interface  Design 

(1 )  Explicit  guidance  should  be  available  defining  the  relationship  between  the  physical  form  of  the  display  and 
its  meaning  with  respect  to  the  plant’s  status. 

Discussion :  NUREG-071 1  specifies  the  need  for  the  HSI  design  to  reflect  HFE  guidelines  in  supporting 
standardization  and  consistency  of  design.  Explicit  guidance  defines  the  relationship  between  the  physical 
form  and  functions  of  the  display  and  its  meaning  with  respect  to  the  plant’s  functions  and  states.  These 
rules  should  consider  the  instrumentation  and  the  data  processing  driving  the  display’s  format.  If  a  single 
display  can  have  more  than  one  interpretation,  it  is  ambiguous  and  can  be  more  easily  misunderstood. 

(2)  The  information  presented  and  its  organization  into  display  pages  should  be  based  on  considering  operators’ 
tasks  when  using  the  displays.  Display  pages  should  include  as  much  information  as  can  be  efficiently 
represented  and  interpreted  to  minimize  the  need  for  operators  to  retrieve  additional  pages. 

Discussion :  Display  pages  should  reflect  operators’  information  needs.  Performance  can  suffer  when 
information  is  missing  (Fujita,  1992;  Elm  and  Woods,  1985)  or  when  the  interface  management  workload 
makes  operators  reluctant  to  retrieve  information  (O’Hara,  Stubler,  and  Nasta,  1 997). 

(3)  When  more  than  one  display  format  is  used  on  a  display  page,  an  evaluation  should  determine  whether  the 
user’s  perception  of  one  format  is  negatively  impacted  by  the  presence  of  the  other  one(s). 

Discussion :  There  has  been  little  research  on  interactions  between  formats;  however,  some  evidence 
indicates  that  unwanted  interactions  can  occur  (Payne  and  Lang,  1995).  For  example,  including  two 
graphic  formats  may  create  an  unintended  emergent  feature.  An  emergent  feature  is  a  high-level,  global 
perceptual  feature  generated  by  the  interactions  among  individual  parts  or  graphical  elements  (e.g.,  lines, 
contours,  and  shapes)  to  produce  perceptual  properties,  such  as  symmetries,  closure,  and  parallelism  (see 
Figure  5.17). 

(4)  The  density  of  information  on  a  display  page  should  be  evaluated  to  ensure  that  important  information  is 
readily  perceived,  and  needed  information  is  rapidly  identified. 

Discussion :  It  is  difficult  to  precisely  specify  the  information  density  of  graphic  displays.  Operators  may 
prefer  displays  of  greater  density  to  minimize  the  need  to  retrieve  information  from  other  pages  (Roth  and 
O’Hara,  1998;  Thomas  et  al.,  1998).  However,  there  are  trade-offs  between  density  and  perceptual  clutter, 
and  a  greater  chance  of  making  errors  when  information  is  not  found.  For  example,  operators  experienced 
difficulty  in  understanding  plant  situations  depicted  in  the  Toshiba  EID  displays  (Itoh,  Sakuma,  and  Monta, 
1995),  partly  because  the  pages  were  too  crowded  with  information.  This  tradeoff  needs  to  be  evaluated  in 
designing  pages. 

(5)  The  organization  scheme  of  display  pages  within  the  network  should  be  readily  apparent  to  operators. 
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Discussion :  In  a  computer-based  HSI,  there  can  be  many  displays;  thus,  the  organization  of  pages  is 
important.  Operators  need  to  know  the  organizational  scheme  of  display  pages  to  locate  and  retrieve  the 
appropriate  ones.  Consideration  may  be  given  to  providing  the  operators  with  variable,  operator-selectable, 
schemes  which  could  enable  them  to  adapt  the  organization  of  data  to  meet  their  current  needs  (Woods  and 
Roth,  1988). 

The  effects  of  instrumentation  failures  on  graphic  displays  should  be  analyzed.  Potential  failure  problems 
should  be  evaluated  with  respect  to  the  following: 

•  Can  operators  detect  a  failure  of  instrumentation? 

•  Can  instrument  failures  result  in  representations  that  are  interpreted  by  operators  as  real  process  failures; 
perhaps  more  importantly,  can  real  process  failures  be  misinterpreted  as  instrument  failures? 

•  If  operators  detect  a  failure,  should  use  of  the  display  be  suspended? 

•  Since  the  display  integrates  many  parameters  into  a  single  display,  what  effect  does  its  loss  have  on 
operations  and  how  effectively  can  operators  transition  to  backup  displays? 

Discussion :  The  failure  modes  of  the  display  and  their  effects  on  plant  operations  must  be  carefully 
examined,  especially  with  complex  graphical  displays.  The  effects  of  instrumentation  failures  need  to  be 
understood  to  ensure  that  they  do  not  lead  to  incorrect  situation  assessment;  i.e.,  operators  mistakenly 
interpreting  a  graphic  change  due  to  an  instrument  failure  as  a  change  in  process  state. 

Access  to  displays  within  a  network  should  be  evaluated  to  ensure  rapid,  efficient  retrieval  of  information 
needed  to  support  operators’  tasks. 

Discussion:  Operators  have  experienced  difficulty  in  gaining  access  to  information  to  support  their  tasks. 
Heslinga  and  Herbert  (1993,  1995)  found  that  information  organized  by  systems  was  sometimes  problematic 
for  operators,  and  required  excessive  interface  management. 

Unwanted  effects  of  integrating  a  new,  novel  graphic  representation  into  a  conventional  HSI  (other  displays, 
other  control  room  HSIs,  and  environmental  considerations  such  as  lighting  levels)  should  be  evaluated  and 
minimized. 

Discussion :  Just  as  graphic  displays  can  interact  within  display  pages,  they  can  interact  with  other 
information  sources  in  the  control  room;  such  interactions  are  not  fully  understood.  Since  there  is  no 
technical  basis  for  guidance  on  these  interactions,  they  should  be  examined  case-by-case  during  the  design 
process. 

The  following  aspects  of  information  system  design  should  be  carefully  analyzed  and  evaluated: 

•  Number  of  VDUs  -  to  ensure  that  the  display  area  is  sufficient  to  show  the  important  information  needed 
by  operators  without  them  having  to  perform  extensive  interface  management 

•  Interface  management  functions  -  to  ensure  that  the  HSI  features  are  easy  to  use  and  provide  explicit 
interface  management  support 

•  Flexibility  of  HSI  and  display  features  and  functions  -  to  ensure  that  the  flexibility  of  the  system  does  not 
unduly  burden  operators,  nor  increase  the  chance  of  misunderstandings  and  errors. 
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Discussion :  The  keyhole  effect  resulting  from  the  operator’s  limited  view  of  plant  information  through 
VDUs,  the  interface  management  tasks,  and  computer-system  flexibility  all  significantly  impact 
performance.  These  factors  have  also  affected  the  design  of  display  pages  and  network  organizations. 

Digital  instrumentation  and  control  systems  typically  provide  much  more  information  than  do  conventional 
systems.  Operators  must  know  what  information  and  controls  are  available  in  the  virtual  workspace,  where 
they  are,  and  how  to  navigate  and  retrieve  them.  Information  is  viewed  through  the  limited  display  area  of 
workstation  VDUs;  this  is  the  so-called  "keyhole  effect"  (Woods  et  al.,  1990).  Consequently,  most 
information  is  hidden  from  view.  Operators  may  have  to  frequently  repeat  interface  management  and 
navigation  tasks  which  can  distract  them  from  their  primary  task  of  monitoring  and  controlling  the  plant. 

For  example,  a  problem  related  to  the  keyhole  effect  is  that  access  to  controls  and  displays  tends  to  be  serial, 
i.e.,  only  a  few  can  be  accessed  at  one  time.  This  contrasts  with  the  parallel  presentation  of  controls  and 
displays  in  conventional  systems,  where  HSIs  are  predominantly  spatially  dedicated  and  their  form  and 
function  does  not  change.  Advanced  HSIs  are  flexible;  they  can  be  configured  and  can  function  in  various 
operating  modes. 

The  flexibility  of  computer-based  HSIs  and  their  lack  of  spatial  dedication  can  be  problematic  for  operators. 
As  discussed  in  O’Hara,  Stubler,  and  Nasta  (1997),  one  reason  may  be  the  reduction  of  the  automaticity  of 
operators’  performance.  When  there  is  a  good  match  between  the  information  available  and  the  operator’s 
mental  model  of  the  task,  performance  becomes  somewhat  automatic,  i.e.,  it  requires  little  attention  and 
cognitive  resources.  When  the  situation  is  unclear,  more  cognitive  resources  are  needed  to  assess  it  and  to 
plan  responses.  With  spatially  dedicated  HSIs,  operators  can  use  automatic  information  processing 
capabilities,  such  as  scanning  and  pattern  recognition,  to  rapidly  assess  situations.  Flexibility  and  lack  of 
spatial  dedication  increases  the  dependence  of  interface  management  tasks  on  controlled  information 
processing.  Flexibility  also  makes  it  easier  for  the  operator  to  mistake  one  display  for  another. 

(10)  Methods  should  be  specified  for  assuring  that  plant  modifications  (such  as  changes  in  instrumentation  or 
systems)  are  incorporated  into  the  display  and  do  not  introduce  inconsistencies  in  how  they  correspond  to 
plant  situations,  or  lead  to  technical  inaccuracies  and,  possibly,  invalid  displays. 

Discussion :  Methods  should  be  specified  for  assuring  that  plant  changes  do  not  introduce  errors  or  changes 
in  the  relationship  between  displays  and  their  representations  of  plant  situations.  For  example,  the 
information  system  may  depend  upon  specific  sensor  data  to  generate  a  graphic  form.  To  maintain  the 
integrity  of  the  information  system,  it  is  critical  that  the  implications  of  changing  any  plant  feature  or 
software  calculation  is  traceable  and  controllable. 

(11)  If  display  formats  are  developed  for  a  generic  plant  design  or  as  an  "off-the-shelf'  product,  any  plant- 
specific  inputs  to  display  characteristics  need  to  be  analyzed  to  ensure  that  the  display  correctly  reflects  the 
relationship  between  changes  in  the  display  format  and  the  changes  in  the  specific  plant  it  is  intended  to 
represent. 

Discussion :  Plant-specific  features  may  affect  the  relationship  between  the  appearance  of  the  display  format 
and  the  plant  conditions  being  represented  in  it.  It  is  important  to  make  sure  the  intended  relationship 
between  the  two  is  preserved  once  plant-specific  details  are  incorporated  into  the  information  system. 

9.4  Training  Program  Development 

(1)  The  knowledge,  skills,  and  abilities  that  the  operators  need  to  use  and  understand  the  information  system 
should  be  specified. 

Discussion :  Research  on  the  use  of  complex  displays,  such  as  the  Rankine  cycle  display  and  DURESS  EID 
display,  illustrated  the  importance  of  training  (Brehmer,  Hill,  and  Brehmer,  1995).  Itoh,  Sakuma,  and 
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Monta  (1995)  found  that  operators  experienced  difficulty  understanding  the  plant  situations  depicted  in  the 
Toshiba  EID  displays.  These  studies  demonstrated  the  importance  of  evaluating  the  relationship  between 
the  representation  in  the  display  and  the  operator’s  mental  model  of  the  domain,  on  the  implications  for 
training. 

Where  the  information  is  sophisticated  and  a  large  amount  is  consolidated  into  one  or  a  few  graphics, 
training  requirements  become  very  important  (Itoh,  Sakuma  and  Monta,  1995).  Operators  need  to 
understand  all  aspects  of  the  display,  and  how  it  reacts  to  various  operational  transients,  accidents,  and 
instrument  failures.  Christoffersen,  Hunter,  and  Vicente  (1995)  stated  that  "...to  experience  the  benefits  of 
EID,  it  seems  likely  that  operators  need  to  be  trained  to  think  functionally  rather  than  procedurally.  It  would 
seem  that  this  would  require  a  fundamental  shift  in  NPP  operation  philosophy"  (p.  143).  They  further 
suggest  that  "...it  may  be  that  operators  have  to  possess  certain  types  of  cognitive  characteristics  that  may  not 
be  considered  in  the  traditional  selection  process  in  the  nuclear  industry"  (p.  143). 

(2)  Operators  should  be  trained  on  the  relationship  between  the  display  form  and  the  plant  states  it  is  intended 
to  represent,  including  failure  modes  and  their  effect  on  graphical  representation. 

Discussion :  Where  large  amounts  of  information  at  various  levels  of  abstraction  are  consolidated  into  one  or 
more  graphic  formats,  training  requirements  become  very  important.  Operators  need  to  understand  all  of 
the  aspects  of  the  display  and  how  it  reacts  to  various  operational  transients,  accidents,  and  instrument 
failures.  Training  should  cover  design  features  of  the  information  system,  relationships  between  graphic 
elements  and  plant  status,  and  the  effects  of  failures. 

(3)  Users  should  be  trained  in  using  the  interface  management  features  of  the  information  system,  including 
navigation  within  and  between  displays,  manipulation  of  on-screen  features  such  as  windows,  and  use  of 
user-definable  characteristics  and  features. 

Discussion :  Successful  use  of  complex  information  systems  depends  to  a  great  extent  on  the  user’s  ability  to 
understand  the  characteristics  of  the  interface  and  effortlessly  use  its  features  (O’Hara  et  al.,  1997).  These 
aspects  of  HSIs  are  often  neglected  in  training,  so  they  are  used  inefficiently. 

(4)  Users  should  be  trained  to  an  acceptable  level  of  proficiency  with  unfamiliar  graphic  displays  before  using 
them  in  the  control  room  to  ensure  that  operators  are  sufficiently  familiar  with  them  to  correctly  assess  their 
meaning. 

Discussion :  The  issue  of  operators’  acceptance  of  a  new  and  different  type  of  display  is  important. 
Introducing  the  displays  initially  in  training  and  then  gradually  into  the  plant  will  help  ensure  their 
acceptance  and  proper  use. 
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The  guidelines  presented  in  this  section  follow  the  characterization  of  information  systems  described  in  Section  4 
of  this  document.  They  also  reflect  the  results  of  the  literature  review  discussed  in  Section  5.  Following  the  HSI 
design  review  procedure  described  in  Part  1  of  NUREG-0700,  Rev.  1 ,  the  first  step  is  to  select  the  subset  of 
guidelines  relevant  to  the  unique  aspects  of  the  particular  design  being  evaluated.  It  is  recognized  that  there  is  a 
wide  range  of  information  system  designs,  and  that  some  may  not  include  all  of  the  characteristics  and  functions 
addressed  in  these  guidelines.  The  reviewer  will  have  to  determine,  case-by-case,  the  importance  of  information 
system  features  that  are  covered  in  the  guidelines,  but  that  are  not  part  of  the  system  being  reviewed. 

Guidelines  were  developed  from  the  findings  and  source  materials  that  were  reviewed  in  Section  5.  They  are 
constructed  in  the  standard  format  adopted  in  NUREG-0700,  Rev.  1  (see  Section  6.2  of  this  report).  The 
guidelines  are  organized  into  the  following  sections: 

•  General  Display  Guidelines 

•  Display  Format 

-  Integral  Formats 

-  Display  Formats 

These  new  guidelines  for  review  of  information  system  designs  will  be  integrated  with  the  information  system 
guidance  in  NUREG-0700,  Rev.  1 . 

Guidelines  may  specify  that  some  identified  aspect  of  an  information  system  needs  to  be  “evaluated.”  General 
approaches  to  evaluation  methods  and  criteria  development  are  defined  in  NUREG-0700.  Since  the  guidance  in 
this  section  will  eventually  be  incorporated  into  that  document,  the  methods  by  which  such  evaluation  should  be 
performed  and  by  which  criteria  should  be  identified  are  not  repeated  here. 

10.1  General  Display  Guidelines 

10.1-1  Correspondence  Mapping 

There  should  be  an  explicit  mapping  between  the  characteristics  and  functions  of  the  system  to  be 
represented  and  the  features  of  the  display  representation,  i.e.,  changes  in  the  appearance  of  the  display 
form  should  have  a  one-to-one  relation  with  the  plant  states  it  represents.  These  changes  should  result 
from  explicit  rules  relating  the  physical  form  of  the  display  and  its  meaning  to  the  plant  state  represented. 
ADDITIONAL  INFORMATION:  Correspondence  mapping  addresses  how  well  the  display 
communicates  meaningful  information  about  the  plant  to  operators.  The  physical  form  and  functions  of 
the  display  should  be  explicitly  tied  to  the  plant’s  functions  and  states.  The  display  form  and  function 
must  consider  the  instrumentation  and  the  data  processing  that  drive  the  display  format.  If  a  single 
display  can  lead  to  more  than  one  interpretation,  it  is  ambiguous  and  can  be  easily  misunderstood. 
Changes  in  the  graphic  display  should  be  unambiguously  related  to  the  plant’s  state.  The  same  graphic 
change  should  not  be  associated  with  more  than  one  interpretation. 

Discussion :  Woods  and  Roth  (1988)  conceptualized  displays  as  involving  three  essential  parts  of  an 
overall  cognitive  system  triad  that  includes  the  world  to  be  acted  on,  the  agents  who  will  act  on  the  world, 
and  the  representation  through  which  the  agents  act  on  the  world.  The  success  of  a  representation  is  a 
function  of  two  types  of  mapping  (Bennett  et  al.,  1997):  correspondence  and  coherence.  (Coherence  is 
addressed  in  Guideline  10.1.3.)  The  importance  of  providing  consistent  one-to-one  mapping  between  the 
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work  domain’s  constraints  and  the  features  of  the  display  has  been  identified  as  an  important  principle 
derived  from  research  on  representing  complex  systems  (Bennett,  Nagy,  and  Flach,  1997;  Vicente  and 
Rasmussen,  1992). 

10.1- 2  Levels  of  Abstraction 

Displays  should  provide  information  at  the  levels  of  abstraction  necessary  to  meet  the  operators’ 
requirements  relative  to  their  task  goals. 

ADDITIONAL  INFORMATION:  Information  should  be  presented  in  accord  with  the  operator’s  goals 
and  the  information  needed  to  address  them. 

Discussion :  This  guideline  reflects  considerations  of  the  information  processing  basis  of  situation 
awareness  support  (Endsley,  1988,  1995).  In  addition,  to  support  knowledge-based  behavior,  Vicente  and 
Rasmussen  (1992)  suggest  that  the  information  system  should  represent  the  work  domain  as  an 
abstraction  hierarchy  to  serve  as  an  externalized  mental  model.  Further,  to  do  this,  Endsley  and  others 
(Heslinga  and  Herbert,  1993  and  1995)  noted  that  consideration  of  tasks  involves  more  than  simply  the 
physical  layout  of  the  system. 

10.1- 3  Coherence  Mapping 

The  characteristics  and  features  of  the  display  used  to  represent  the  process  should  .be  readily  perceived 
and  interpreted  by  the  operator. 

ADDITIONAL  INFORMATION:  Coherence  mapping  addresses  how  comprehensible  the  representation 
is  to  the  operator.  Unambiguous  relationships  between  the  display  and  the  process  are  of  little  value  if 
they  also  are  not  readily  perceived  by  the  operators  and  easily  understood. 

Discussion :  See  discussion  of  Guideline  10.1-1 . 

10.1- 4  Understandability  of  Higher-Level  Information 

The  methods  by  which  lower-level  data  are  analyzed  to  produce  higher-level  information  and  graphical 
elements  should  be  understandable  to  users. 

ADDITIONAL  INFORMATION:  Users  must  be  able  to  judge  the  acceptability  of  higher-level 
information  and  how  it  relates  to  lower-level  information. 

Discussion :  To  maintain  their  role  as  system  supervisors,  operators  need  to  be  able  to  understand  and 
evaluate  the  appropriateness  and  validity  of  the  information  being  displayed. 

10.1- 5  User  Verification  of  Higher-Level  Information 

Operators  should  have  access  to  the  rules  or  computations  that  link  process  parameters  and  graphical 
features,  and  to  an  explanation  of  how  the  information  system  produces  higher-level  information. 
ADDITIONAL  INFORMATION:  When  graphical  features  change  in  ways  not  completely  understood  by 
operators,  they  should  be  able  to  access  the  rules  that  produce  the  graphic  forms.  Operators  should  be  able 
to  review  any  analysis  performed  by  the  information  system. 

Discussion :  To  maintain  their  role  as  system  supervisors,  operators  need  to  be  able  to  access  information 
enabling  them  to  determine  the  appropriateness  and  validity  of  information  displayed.  Access  to  the  rules 
and  computations  that  relate  the  system  and  the  display  (the  correspondence  map)  should  be  available  to 
operators  to  support  situation  assessment.  In  general,  computer-based  systems  often  are  not  sufficiently 
observable.  That  is,  they  do  not  make  clear  their  analysis  basis  and  do  not  have  adequate  communication 
facilities  to  enable  operators  to  verify  system  performance  (Dien  and  Montmayeul,  1995;  IAEA,  1994; 
Malin  et  al.,  1992a;  Roth,  Bennet,  and  Woods,  1987). 

10.1- 6  Salience  Levels 

The  salience  of  graphic  features  should  reflect  the  importance  of  the  information. 
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ADDITIONAL  INFORMATION:  The  most  salient  features  of  a  graphic  display  should  be  those  aspects 
of  the  representation  that  are  most  important.  Less  important  information  should  not  be  more 
perceptually  salient  than  more  important  information. 

Discussion :  This  guideline  reflects  considerations  of  the  information  processing  basis  of  situation 
awareness  support  (Endsley,  1988,  1995),  and  research  by  Bennett,  Nagy,  and  Flach  (1997)  on  the 
salience  of  emergent  features.  Situation  assessment  depends  on  activating  the  appropriate  mental 
representation  for  the  situation;  thus,  the  critical  cues  to  such  activation  should  be  determined,  and 
displays  should  make  these  cues  salient. 

10.1- 7  Display  of  Goal  Status 

The  information  system  should  provide  for  global  situation  awareness  (i.e.,  an  overview  of  the  status  of  all 
the  operator’s  goals  at  all  times)  as  well  as  supplying  details  about  the  current  specific  goal. 

ADDITIONAL  INFORMATION:  Situation  assessment  can  suffer  when  operators  focus  on  some 
information  and  fail  to  attend  to  other  important  data. 

Discussion :  This  guideline  reflects  considerations  of  the  information  processing  basis  of  situation 
awareness  support  (Endsley,  1988,  1995). 

10.1- 8  Display  of  Future  Status 

The  information  system  should  support  the  user’s  ability  to  project  future  states  of  the  system  when  this  is 
required  to  safely  operate  the  plant. 

ADDITIONAL  INFORMATION:  Situation  assessment  involves  not  only  understanding  the  current  state 
of  the  plant,  but  projecting  its  future  state.  Displays  such  as  trend  graphs  can  support  these  projections. 
Discussion:  This  guideline  reflects  considerations  of  the  information  processing  basis  of  situation 
awareness  support  (Endsley,  1988,  1995). 

10.1- 9  Minimize  Attention  Shifts 

Information  needed  by  the  operator  to  accomplish  a  task  should  be  grouped  and  perceptually  related,  when 
possible. 

ADDITIONAL  INFORMATION:  To  minimize  the  disadvantages  of  divided  attention,  the  number  of 
attention  shifts  should  be  minimized,  both  within  a  display  page  and  between  them. 

Discussion :  This  guideline  reflects  considerations  of  the  information  processing  basis  of  situation 
awareness  support  (Endsley,  1988, 1995).  Interface  management  tasks  necessary  to  retrieve  displays  and 
to  shift  between  them  are  distracting,  and  can  degrade  performance.  Such  distractions  should  be 
minimized. 

10.1- 10  Analytical  Redundancy 

Analytical  redundancy  should  be  considered  to  help  ensure  the  appropriateness  of  displayed  values. 
ADDITIONAL  INFORMATION:  Analytical  redundancy  is  the  calculation  of  expected  parameter  values 
using  a  model  of  system  performance.  The  expected  value  is  then  represented  in  the  display,  along  with 
the  actual  value.  Deviations  between  the  two  indicate  some  disturbance  or  abnormality  of  the  system. 
Discussion :  Vicente  and  Wang  (1996)  found  that  fault  detection  time  was  reduced  and  the  ability  of  study 
participants  to  compensate  for  the  fault  was  improved  when  a  display  had  analytical  redundancy. 

10.1- 11  Failure  Recognition 

Information  system  failures  (due  to  sensors,  instruments,  and  components)  should  result  in  distinct  display 
changes  which  directly  indicate  that  depicted  plant  conditions  are  invalid. 

ADDITIONAL  INFORMATION:  The  information  system  should  be  designed  so  that  failures  in 
instrumentation  are  readily  recognized  by  operators.  This  may  be  more  difficult  to  determine  in  complex 
graphics,  and  thus,  should  be  carefully  evaluated. 
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Discussion :  Moray  et  al.  (1994)  noted  that  graphical  interfaces  are  "extremely  vulnerable"  to  this  type  of 
failure  and  that  understanding  their  failure  modes  is  as  necessary  as  understanding  their  design  for  normal 
conditions. 

10.1- 12  Navigational  Links  to  Related  Information 

Navigational  links  to  and  from  high-level  and  lower-levels  of  information  and  to  reference  and  supporting 
information  should  be  provided  when  needed  for  operators’  tasks. 

Discussion :  Navigation  to  and  from  related  information  can  be  time  consuming,  distracting,  and  error 
prone.  Computer  support,  such  as  hyperlinks,  for  these  activities  can  reduce  the  workload. 

10.2  Display  Format 

10.2.1  Integral  Formats 

10.2.1- 1  Appropriate  Use  of  Integral  Displays 

Integral  formats  should  be  used  to  communicate  high-level,  status-at-a-glance  information  where  users 
may  not  need  information  on  individual  parameters  to  interpret  the  display. 

ADDITIONAL  INFORMATION:  Since  integral  displays  do  not  display  individual  parameters,  they  are 
most  appropriate  for  general  status  monitoring. 

Discussion :  NPP  operators  almost  always  need  information  on  individual  parameters,  except  for  general 
monitoring.  Integral  displays  best  serve  this  purpose  because  they  do  not  show  individual  parameters. 

10.2.1- 2  Reference  Aids  for  Object  Displays 

A  perceptually  distinct  reference  aid  should  be  provided  in  an  object  display  to  support  operators  in 
recognizing  abnormalities  in  the  object’s  characteristics. 

ADDITIONAL  INFORMATION:  When  a  change  in  an  object’s  characteristics  (e.g.,  its  shape)  is  the 
perceptual  feature  that  indicates  a  fault  or  abnormal  condition,  perceptual  cues  can  assist  operators  in 
detecting  the  change.  If  shape  is  used,  the  graphic  display  should  include  the  normal  reference  point  to 
which  operators  can  compare  the  current  shape.  Reference  points  are  especially  useful  when  the 
abnormality  is  slow  to  evolve,  and  the  integral  object  is  changing  slowly. 

Discussion :  There  is  little  research  guidance  on  how  much  a  graphic  form  must  change  to  be  perceptually 
recognized.  Thus,  if  a  referent  is  not  present,  it  may  be  difficult  to  ensure  that  a  change  is  recognized. 
Recognition  is  supported  by  including  the  comparison  referent. 

10.2.2  Configural  Display  Formats 

10.2.2- 1  Appropriate  Use  of  Configural  Displays 

Configural  formats  should  be  used  when  operators  must  rapidly  transition  between  high-level  functional 
information  and  specific  parameter  values. 

ADDITIONAL  INFORMATION:  Configural  displays  provide  lower-level  information,  such  as  parameter 
values,  and  higher-level  information  conveyed  through  emergent  features.  Since  both  are  present  in  a 
single  display,  operators  can  easily  move  between  them. 

Discussion :  The  value  of  configural  features  is  their  ability  to  convey  multiple  levels  of  information  in  a 
single  display.  Therefore,  when  operators  must  be  cognizant  of  more  than  one  level  of  abstraction  of 
information,  configural  displays  are  preferred. 
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10.2.2- 2  Representation  of  Emergent  Features 

The  display  elements  should  be  organized  so  that  the  emergent  features  that  arise  from  their  interaction 
correspond  to  meaningful  information  about  the  process  or  system,  e.g.,  when  the  aspect  of  the  system 
represented  by  the  emergent  is  disturbed,  the  disturbance  is  visible  in  the  emergent  feature. 

ADDITIONAL  INFORMATION:  An  emergent  feature  is  a  high-level,  global  perceptual  feature 
generated  by  interactions  among  individual  parts  or  graphical  elements  of  a  display  (e.g.,  lines,  contours, 
and  shapes)  to  produce  perceptual  properties,  such  as  symmetries,  closure,  and  parallelism.  Displays 
cannot  always  be  organized  to  provide  emergent  features,  but  they  should  be  considered  where  feasible. 
Discussion :  This  guideline  is  was  developed  by  Bennett,  Nagy,  and  Flach  (1997)  from  reviewing  research 
on  configural  displays. 

10.2.2- 3  Levels  of  Emerging  Features 

The  emergent  features  or  patterns  within  the  display  should  be  nested  (from  global  to  local)  in  a  way  that 
reflects  the  hierarchical  structure  of  the  process. 

ADDITIONAL  INFORMATION:  High-order  aspects  of  the  process  (e.g.,  at  the  level  of  functional 
purpose  or  abstract  function)  should  be  reflected  in  global  display  features;  lower-order  aspects  of  the 
process  (e.g.,  functional  organization)  should  be  reflected  in  local  display  features. 

Discussion :  This  guideline  is  was  developed  by  Bennett,  Nagy,  and  Flach  (1997),  who  reviewed  research 
on  configural  displays. 

10.2.2- 4  Salience  of  Emerging  Features 

Each  emergent  feature  should  be  clearly  distinguishable  for  other  emergent  features  and  from  information 
on  individual  parameters. 

Discussion :  This  guideline  is  was  developed  by  Bennett,  Nagy,  and  Flach  (1997)  based  on  a  review  of 
research  on  configural  displays. 

10.2.2- 5  Reference  Aids  for  Configural  Displays 

A  perceptually  distinct  reference  aid  should  be  provided  in  a  configural  display  to  support  operators  in 
recognizing  abnormalities  in  emergent  features. 

ADDITIONAL  INFORMATION:  When  a  change  in  an  object’s  characteristics  (e.g.,  its  shape)  is  the 
perceptual  feature  that  indicates  a  fault  or  abnormal  condition,  perceptual  cues  can  assist  operators  in 
detecting  the  change.  If  shape  is  used,  the  display  graphic  should  include  the  normal  reference  point 
against  which  operators  can  compare  the  current  one.  Reference  points  are  especially  useful  when  the 
abnormality  is  slow  to  evolve,  and  the  integral  object  is  slowly  changing. 

Discussion :  See  discussion  of  Guideline  10.2.1-2. 

10.2.2- 6  Representation  of  Individual  Parameters 

Each  relevant  process  parameter  should  be  represented  by  a  perceptually  distinct  element  within  the 
display. 

Discussion :  Since  configural  displays  provide  both  high-  and  low-level  information,  all  relevant  lower- 
level  parameters  should  be  given.  This  will  minimize  the  need  for  operators  to  retrieve  supplemental 
information  directly  relevant  to  the  configural  display  from  another  one. 

10.2.2- 7  Use  of  Lower-Level  Information 

The  display  should  support  the  user  in  performing  tasks  requiring  lower-level  information. 

ADDITIONAL  INFORMATION:  When  the  operator  must  perform  tasks  using  lower  level  information, 
the  display  should  provide  such  support.  For  example,  if  precise  information  about  a  variable  is  desirable, 
then  a  scale  or  digital  information  should  be  provided. 
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Discussion:  Providing  information  to  support  detailed  use  of  lower-level  information  will  minimize  the 
chance  for  error  in  deriving  the  lower-level  information  from  the  display,  and  minimize  the  need  to 
retrieve  supplemental  information  from  another  display. 

10.2.2-8  Complexity 

The  emergent  features  and  their  interactions  should  not  be  so  complex  as  to  be  susceptible  to 
misinterpretation. 

ADDITIONAL  INFORMATION:  The  value  of  emergent  features  is  that  they  provide  a  direct  perception 
of  higher-level  information.  They  substitute  perception  for  mental  calculation.  The  shift  toward 
perceptual  cognition  requires  careful  design,  so  that  misunderstandings  are  unlikely  to  occur. 

Discussion :  Configural  displays  can  get  complex  and  can  be  misunderstood  by  operators  (Itoh,  Sakuma, 
andMonta,  1995). 
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Analytical  Redundancy  -  The  calculation  of  expected  parameter  values  using  a  model  of  system  performance. 

Coherence  Mapping  -  A  map  between  the  features  in  the  representation  and  the  physical  and  cognitive 
characteristics  of  the  operator  (how  comprehensible  the  representation  is  to  the  operator). 

Configural  Display  -A  display  in  which  information  dimensions  are  uniquely  represented,  but  where  new 
emergent  properties  are  created  from  interactions  between  the  dimensions.  Configural  display  representations 
often  use  simple  graphic  forms,  such  as  a  polygon. 

Correspondence  Mapping  -  A  map  between  the  properties  and  characteristics  of  the  system  to  be  represented  and 
the  features  in  the  representation  (how  well  the  display  communicates  meaningful  information  about  the  plant  to 
operators). 

Display  Devices  -  The  media  used  to  present  information  to  personnel,  including  meters,  gauges,  VDUs,  and  hard¬ 
copy  display  devices  (printers  and  plotters). 

Display  Element  -  The  basic  building  blocks  used  to  make  up  display  formats,  such  as  abbreviations,  labels,  icons, 
symbols,  coding,  and  highlighting. 

Display  Format  -  The  general  class  of  information  presentation.  Examples  of  general  classes  are  continuous  text 
(such  as  a  procedure  display),  mimics  and  piping  and  instrumentation  diagram  (P&ID)  displays,  trend  graphs,  and 
flowcharts. 

Display  Network-  An  group  of  display  pages  within  an  information  system  and  their  organizational  structure. 

Display  Page  -  A  defined  set  of  information  that  is  intended  to  be  displayed  as  a  single  unit.  Typical  NPP  display 
pages  may  combine  several  different  formats  on  a  single  VDU  screen,  such  as  putting  bar  charts  and  digital 
displays  in  a  graphic  P&ID  format.  Display  pages  typically  have  a  label  and  designation  within  the  computer 
system  so  they  can  be  accessed  by  operators  as  a  single  "display.” 

Emergent  Feature  -  A  high-level,  global  perceptual  feature  produced  by  the  interactions  among  individual  parts  or 
graphical  elements  of  a  display  (e.g.,  lines,  contours,  and  shapes). 

Information  System  -  Those  aspects  of  the  HSI  that  provide  information  on  the  plant’s  systems  to  the  operator. 

Integral  Display  -  A  display  that  depicts  the  integration  of  information  in  such  a  way  that  the  individual 
parameters  used  to  generate  the  display  are  not  explicitly  represented  in  it. 

Level  of  Abstraction  -  A  hierarchy  consisting  of  levels  increasing  in  abstraction: 

-  Physical  form  -  the  appearance  and  spatial  location  of  the  components 

-  Physical  function  -  the  characteristics  of  the  components  and  their  interconnections 

-  Generalized  function  -  the  basic  functions  a  system  was  designed  to  achieve 

-  Abstract  function  -  the  causal  structure  of  the  process  in  terms  of  mass,  energy,  information  or  value  flows 

-  Functional  purpose  -  the  purpose  for  which  the  system  was  designed;  the  functional  characteristics  of  the 
plant  as  opposed  to  physical  characteristics. 
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Object  Display  -  A  type  of  integral  display  that  uses  a  geometric  object  to  represent  parameter  values  graphically, 
but  where  the  individual  information  dimensions  or  data  contributing  to  the  object  are  not  displayed. 

Separable  Display  -  Each  process  parameter  is  presented  individually  and  no  relationships  between  the  parameters 
are  shown  by  the  representation  itself  The  key  aspect  of  separable  displays  is  not  that  individual  parameters  are 
presented,  but  that  no  interaction  or  relationship  between  them  is  perceived. 
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